Correlating user reaction with at least an aspect associated with an augmentation of an augmented view

ABSTRACT

Computationally implemented methods and systems include detecting one or more user reactions of a user in response to a display to the user of an augmented view of an actual scene from a real environment, the augmented view that was displayed including one or more augmentations, and correlating the detected one or more user reactions with at least one or more aspects associated with the one or more augmentations that were included in the augmented view that was presented. In addition to the foregoing, other aspects are described in the claims, drawings, and text.

If an Application Data Sheet (ADS) has been filed on the filing date ofthis application, it is incorporated by reference herein. Anyapplications claimed on the ADS for priority under 35 U.S.C. §§119, 120,121, or 365(c), and any and all parent, grandparent, great-grandparent,etc. applications of such applications, are also incorporated byreference, including any priority claims made in those applications andany material incorporated by reference, to the extent such subjectmatter is not inconsistent herewith.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is related to and/or claims the benefit of theearliest available effective filing date(s) from the following listedapplication(s) (the “Priority Applications”), if any, listed below(e.g., claims earliest available priority dates for other thanprovisional patent applications or claims benefits under 35 USC §119(e)for provisional patent applications, for any and all parent,grandparent, great-grandparent, etc. applications of the PriorityApplication(s)). In addition, the present application is related to the“Related Applications,” if any, listed below.

PRIORITY APPLICATIONS

-   -   For purposes of the USPTO extra-statutory requirements, the        present application constitutes a continuation-in-part of U.S.        patent application Ser. No. 13/711,095, entitled SYSTEMS AND        METHODS FOR OBTAINING AND USING AUGMENTATION DATA AND FOR        SHARING USAGE DATA, naming GENE FEIN; ROYCE A. LEVIEN;        RICHARD T. LORD; ROBERT W. LORD; MARK A. MALAMUD; JOHN D.        RINALDO, JR.; CLARENCE T. TEGREENE as inventors, filed 11, Dec.,        2012 with attorney docket no. 0412-003-004-000001, which is        currently co-pending or is an application of which a currently        co-pending application is entitled to the benefit of the filing        date, and which is a continuation of U.S. patent application        Ser. No. 13/709,465, entitled SYSTEMS AND METHODS FOR OBTAINING        AND USING AUGMENTATION DATA AND FOR SHARING USAGE DATA, naming        GENE FEIN; ROYCE A. LEVIEN; RICHARD T. LORD; ROBERT W. LORD;        MARK A. MALAMUD; JOHN D. RINALDO, JR.; CLARENCE T. TEGREENE as        inventors, filed 10, Dec., 2012 with attorney docket no.        0412-003-004-000000.    -   For purposes of the USPTO extra-statutory requirements, the        present application constitutes a continuation-in-part of U.S.        patent application Ser. No. 13/690,003, entitled SYSTEMS AND        METHODS FOR SHARING AUGMENTATION DATA, naming GENE FEIN;        ROYCE A. LEVIEN; RICHARD T. LORD; ROBERT W. LORD; MARK A.        MALAMUD; JOHN D. RINALDO, JR.; CLARENCE T. TEGREENE as        inventors, filed 30, Nov., 2012 with attorney docket no.        0412-003-003-000001, which is currently co-pending or is an        application of which a currently co-pending application is        entitled to the benefit of the filing date, and which is a        continuation of U.S. patent application Ser. No. 13/689,372,        entitled SYSTEMS AND METHODS FOR SHARING AUGMENTATION DATA,        naming GENE FEIN; ROYCE A. LEVIEN; RICHARD T. LORD; ROBERT W.        LORD; MARK A. MALAMUD; JOHN D. RINALDO, JR.; CLARENCE T.        TEGREENE as inventors, filed 29, Nov., 2012 with attorney docket        no. 0412-003-003-000000.    -   For purposes of the USPTO extra-statutory requirements, the        present application constitutes a continuation-in-part of U.S.        patent application Ser. No. 13/673,070, entitled PRESENTING AN        AUGMENTED VIEW IN RESPONSE TO ACQUISITION OF DATA INFERRING USER        ACTIVITY, naming GENE FEIN; ROYCE A. LEVIEN; RICHARD T. LORD;        ROBERT W. LORD; MARK A. MALAMUD; JOHN D. RINALDO, JR.;        CLARENCE T. TEGREENE as inventors, filed 9, Nov., 2012 with        attorney docket no. 0412-003-002-000001, which is currently        co-pending or is an application of which a currently co-pending        application is entitled to the benefit of the filing date, and        which is a continuation of U.S. patent application Ser. No.        13/672,575, entitled PRESENTING AN AUGMENTED VIEW IN RESPONSE TO        ACQUISITION OF DATA INFERRING USER ACTIVITY, naming GENE FEIN;        ROYCE A. LEVIEN; RICHARD T. LORD; ROBERT W. LORD; MARK A.        MALAMUD; JOHN D. RINALDO, JR.; CLARENCE T. TEGREENE as        inventors, filed 8, Nov., 2012 with attorney docket no.        0412-003-002-000000.    -   For purposes of the USPTO extra-statutory requirements, the        present application constitutes a continuation-in-part of U.S.        patent application Ser. No. 13/648,012, entitled FORMATTING OF        ONE OR MORE PERSISTENT AUGMENTATIONS IN AN AUGMENTED VIEW IN        RESPONSE TO MULTIPLE INPUT FACTORS, naming GENE FEIN; ROYCE A.        LEVIEN; RICHARD T. LORD; ROBERT W. LORD; MARK A. MALAMUD;        JOHN D. RINALDO, JR.; CLARENCE T. TEGREENE as inventors, filed        9, Oct., 2012 with attorney docket no. 0412-003-001-000001,        which is currently co-pending or is an application of which a        currently co-pending application is entitled to the benefit of        the filing date, and which is a continuation of U.S. patent        application Ser. No. 13/646,147, entitled FORMATTING OF ONE OR        MORE PERSISTENT AUGMENTATIONS IN AN AUGMENTED VIEW IN RESPONSE        TO MULTIPLE INPUT FACTORS, naming GENE FEIN; ROYCE A. LEVIEN;        RICHARD T. LORD; ROBERT W. LORD; MARK A. MALAMUD; JOHN D.        RINALDO, JR.; CLARENCE T. TEGREENE as inventors, filed 5, Oct.,        2012 with attorney docket no. 0412-003-001-000000.

RELATED APPLICATIONS

None

The United States Patent Office (USPTO) has published a notice to theeffect that the USPTO's computer programs require that patent applicantsreference both a serial number and indicate whether an application is acontinuation, continuation-in-part, or divisional of a parentapplication. Stephen G. Kunin, Benefit of Prior-Filed application, USPTOOfficial Gazette Mar. 18, 2003. The USPTO further has provided forms forthe Application Data Sheet which allow automatic loading ofbibliographic data but which require identification of each applicationas a continuation, continuation-in-part, or divisional of a parentapplication. The present Applicant Entity (hereinafter “Applicant”) hasprovided above a specific reference to the application(s) from whichpriority is being claimed as recited by statute. Applicant understandsthat the statute is unambiguous in its specific reference language anddoes not require either a serial number or any characterization, such as“continuation” or “continuation-in-part,” for claiming priority to U.S.patent applications. Notwithstanding the foregoing, Applicantunderstands that the USPTO's computer programs have certain data entryrequirements, and hence Applicant has provided designation(s) of arelationship between the present application and its parentapplication(s) as set forth above and in any ADS filed in thisapplication, but expressly points out that such designation(s) are notto be construed in any way as any type of commentary and/or admission asto whether or not the present application contains any new matter inaddition to the matter of its parent application(s). If the listings ofapplications provided above are inconsistent with the listings providedvia an ADS, it is the intent of the Applicant to claim priority to eachapplication that appears in the Priority Applications section of the ADSand to each application that appears in the Priority Applicationssection of this application. All subject matter of the PriorityApplications and the Related Applications and of any and all parent,grandparent, great-grandparent, etc. applications of the PriorityApplications and the Related Applications, including any priorityclaims, is incorporated herein by reference to the extent such subjectmatter is not inconsistent herewith.

SUMMARY

In one or more various aspects, a method includes but is not limited todetecting one or more user reactions of a user in response to a displayto the user of an augmented view of an actual scene from a realenvironment, the augmented view that was displayed including one or moreaugmentations, and correlating the detected one or more user reactionswith at least one or more aspects associated with the one or moreaugmentations that were included in the augmented view that waspresented. In some implementations, at least one of the detecting orcorrelating being performed by a machine, article of manufacture, orcomposition of matter. In addition to the foregoing, other methodaspects are described in the claims, drawings, and text forming a partof the disclosure set forth herein.

In one or more various aspects, one or more related systems may beimplemented in machines, compositions of matter, or manufactures ofsystems, limited to patentable subject matter under 35 U.S.C. 101. Theone or more related systems may include, but are not limited to,circuitry and/or programming for effecting the herein-referenced methodaspects. The circuitry and/or programming may be virtually anycombination of hardware, software, and/or firmware configured to effectthe herein-referenced method aspects depending upon the design choicesof the system designer, and limited to patentable subject matter under35 USC 101.

In one or more various aspects, a system includes, but is not limitedto, means for detecting one or more user reactions of a user in responseto a display to the user of an augmented view of an actual scene from areal environment, the augmented view that was displayed including one ormore augmentations, and means for correlating the detected one or moreuser reactions with at least one or more aspects associated with the oneor more augmentations that were included in the augmented view that waspresented. In addition to the foregoing, other system aspects aredescribed in the claims, drawings, and text forming a part of thedisclosure set forth herein.

In one or more various aspects, a system includes, but is not limitedto, circuitry for detecting one or more user reactions of a user inresponse to a display to the user of an augmented view of an actualscene from a real environment, the augmented view that was displayedincluding one or more augmentations, and circuitry for correlating thedetected one or more user reactions with at least one or more aspectsassociated with the one or more augmentations that were included in theaugmented view that was presented. In addition to the foregoing, othersystem aspects are described in the claims, drawings, and text forming apart of the disclosure set forth herein.

In one or more various aspects, a computer program product, comprising asignal bearing non-transitory storage medium, bearing one or moreinstructions including, but not limited to, detecting one or more userreactions of a user in response to a display to the user of an augmentedview of an actual scene from a real environment, the augmented view thatwas displayed including one or more augmentations, and correlating thedetected one or more user reactions with at least one or more aspectsassociated with the one or more augmentations that were included in theaugmented view that was presented. In addition to the foregoing, othercomputer program product aspects are described in the claims, drawings,and text forming a part of the disclosure set forth herein.

In addition to the foregoing, various other method and/or system and/orprogram product aspects are set forth and described in the teachingssuch as text (e.g., claims and/or detailed description) and/or drawingsof the present disclosure.

The foregoing is a summary and thus may contain simplifications,generalizations, inclusions, and/or omissions of detail; consequently,those skilled in the art will appreciate that the summary isillustrative only and is NOT intended to be in any way limiting. Otheraspects, features, and advantages of the devices and/or processes and/orother subject matter described herein will become apparent by referenceto the detailed description, the corresponding drawings, and/or in theteachings set forth herein.

BRIEF DESCRIPTION OF THE FIGURES

For a more complete understanding of embodiments, reference now is madeto the following descriptions taken in connection with the accompanyingdrawings. The use of the same symbols in different drawings typicallyindicates similar or identical items, unless context dictates otherwise.The illustrative embodiments described in the detailed description,drawings, and claims are not meant to be limiting. Other embodiments maybe utilized, and other changes may be made, without departing from thespirit or scope of the subject matter presented here.

FIGS. 1A and 1B show the front and back views of an augmented reality(AR) device that is in the form of a smartphone 10.

FIGS. 2A and 2B show the front and back views of an augmented reality(AR) device that is in the form of a tablet computer 20.

FIGS. 3A and 3B show different views of an augmented reality (AR) devicethat is in the form of a goggle 30.

FIGS. 4A and 4B show different views of an augmented reality (AR) devicethat is in the form of a goggle 40.

FIG. 5A shows a user 52 using the tablet computer 20 of FIGS. 2A and 2Bin order to view two different scenes in the real environment 50 at twodifferent points or increments in time.

FIG. 5B shows a side view of the user 52 of FIG. 5A using the tabletcomputer 20.

FIGS. 6A, 6B, 6C, 6D, 6E, 6F, 6G, 6H, 6I, 6K, and 6M show exemplaryactual and augmented views of various scenes from the real environment.

FIG. 7A shows a block diagram of particular implementation of an ARdevice.

FIG. 7B shows a block diagram of yet another implementation of an ARdevice.

FIG. 8A shows another perspective of the user reaction ascertainingmodule 102* of FIGS. 7A and 7B (e.g., the user reaction ascertainingmodule 102′ of FIG. 7A or the user reaction ascertaining module 102″ ofFIG. 7B) in accordance with various implementations.

FIG. 8B shows the user reaction associating module 104* of FIGS. 7A and7B (e.g., the user reaction associating module 104′ of FIG. 7A or theuser reaction associating module 104″ of FIG. 7B) in accordance withvarious implementations.

FIG. 8C shows another perspective of the user interface 110 of FIGS. 7Aand 7B.

FIG. 8D shows another perspective of the one or more sensors 120 ofFIGS. 7A and 7B.

FIG. 9 is a high-level logic flowchart of a process, e.g., operationalflow 900, according to some embodiments.

FIG. 10A is a high-level logic flowchart of a process depictingalternate implementations of the user reaction detecting operation 902of FIG. 9.

FIG. 10B is a high-level logic flowchart of a process depictingalternate implementations of the user reaction detecting operation 902of FIG. 9.

FIG. 10C is a high-level logic flowchart of a process depictingalternate implementations of the user reaction detecting operation 902of FIG. 9.

FIG. 10D is a high-level logic flowchart of a process depictingalternate implementations of the user reaction detecting operation 902of FIG. 9.

FIG. 11A is a high-level logic flowchart of a process depictingalternate implementations of the user reaction correlating operation 904of FIG. 9.

FIG. 11B is a high-level logic flowchart of a process depictingalternate implementations of the user reaction correlating operation 904of FIG. 9.

FIG. 11C is a high-level logic flowchart of a process depictingalternate implementations of the user reaction correlating operation 904of FIG. 9.

FIG. 11D is a high-level logic flowchart of a process depictingalternate implementations of the user reaction correlating operation 904of FIG. 9.

FIG. 11E is a high-level logic flowchart of a process depictingalternate implementations of the user reaction correlating operation 904of FIG. 9.

FIG. 12 is another high-level logic flowchart of a process, e.g.,operational flow 1200, according to some embodiments.

FIG. 13 depicts alternate implementations of the operational flow 1200of FIG. 12.

FIG. 14 is another high-level logic flowchart of a process, e.g.,operational flow 1400, according to some embodiments.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawings, which form a part hereof. In the drawings,similar symbols typically identify similar or identical components oritems, unless context dictates otherwise. The illustrative embodimentsdescribed in the detailed description, drawings, and claims are notmeant to be limiting. Other embodiments may be utilized, and otherchanges may be made, without departing from the spirit or scope of thesubject matter presented here.

Throughout this application, examples and lists are given, withparentheses, the abbreviation “e.g.,” or both. Unless explicitlyotherwise stated, these examples and lists are merely exemplary and arenon-exhaustive. In most cases, it would be prohibitive to list everyexample and every combination. Thus, smaller, illustrative lists andexamples are used, with focus on imparting understanding of the claimterms rather than limiting the scope of such terms.

The evolution of personal computing technology over the past 40 yearshas been simply breathtaking, evolving from clunky large personaldesktop computing devices with poor resolution television monitors andlimited functionality to sleek and powerful mobile computing devicessuch as tablet computers and smartphones. As the personal computingtechnology continues to evolve a number of promising and exciting newtechnologies have emerged that are rapidly being developed forwidespread personal use. One of the most promising new technologies isAugmented Reality (or simply “AR”).

Wikipedia™ defines Augmented Reality as “a live, direct or indirect,view of a physical, real-world environment whose elements are augmentedby computer-generated sensory input such as sound, video, graphics orGPS data.” In order to facilitate understanding of the various concepts,processes, and systems to be discussed herein, certain basic terms andphrases will now be introduced and elucidated. For example, in thefollowing the terms “augmentation,” “augmented view,” “actual view,”“scene from a real environment,” or variations thereof, are repeatedlyrecited. For purposes of the following, the phrase “scene from a realenvironment” will be in reference to an actual or true (visual) scenefrom an actual physical environment (as opposed to a virtual environmentor world) in the proximate vicinity of an AR system and/or the user ofthe AR system (herein “AR device user”). For purposes of the followingdescription, the phrase “actual view” is in reference to a true orunmodified (or substantially true or unmodified) view of a scene fromthe real environment. The phrase “augmented view,” in contrast, is inreference to a view of an actual scene from the real environment thathas been augmented (e.g., modified) and that may be presented (e.g.,displayed or transmitted) through an AR system. An “augmentation” is anymodification, revision, or addition that may be included in an augmentedview of a scene from the real environment and that may not be present inthe actual view of the scene from the real environment.

There are at least two types of computing devices that can be used toimplement AR technology: “specifically-designed” AR systems and“nonspecifically-designed” AR systems. Nonspecifically-designed ARsystems are general purpose computing systems or devices that can beconfigured to implement AR functionalities by executing, for example, ARsoftware applications. Examples of such devices include, for example,personal mobile computing/communication devices such as tablet computersand smartphones. In contrast, specifically-designed AR systems aresystems or devices that have been specifically designed to implement ARfunctionalities. Specifically-designed AR systems may come in a varietyof forms but are most commonly in the form of a head-mounted display(HMD) such as in the form of eyeglasses, goggles, helmet, and so forth.These devices are sometimes referred to as “wearable computing devices.”Typically these wearable computing devices will include one or morefeatures that allows the user to wear the device on his/or her head(e.g., a coupling device for coupling the AR device to a user's head).Such features include, for example, a strap, a pair of earpieces ortemple pieces (e.g., the parts of goggles or glasses that are attachedto the frame of, for example, a glasses and that extend out to the earsof the user holding the eyeglasses frame to the face of the user—seeexample 42 a or 42 b of FIG. 4A, or in the case of a helmet the helmetstructure itself). Alternatively, a wearable computing device may bemade wearable by having a feature (e.g., one or more clips or hooks)that allows it to be attached or clipped onto the frame of a pair ofglasses or goggles thus allowing the AR device to be coupled to theuser's head. An example of such a feature may be found in the form of aclip 47 a is illustrated in FIG. 4A.

All of these devices (e.g., specifically-designed AR systems andnonspecifically-designed AR systems) will usually have certain commoncomponents including one or more cameras (e.g., digital, web, and/or HDcameras), one or more displays (e.g., LCD displays or see-throughdisplays), and logic for processing data generated by the one or morecameras and/or for generating and merging computer generated data orimages with actual views or images of scenes from real world environmentto generate augmented views of the scenes of the real world environment.Although most, if not all, current AR systems will contain these samebasic components (e.g., camera, display, etc.), they can, however, takeon a wide range of form factors as briefly described above (e.g., tabletcomputer, goggles, and so forth).

FIGS. 1A and 1B, 2A and 2B, 3A and 3B, and 4A and 4B illustrate some ofthe form factors that current AR devices can take on. For example, FIG.1A depicts the front-side 12 (e.g., the side of a personalcomputing/communication device that a display is located or disposedon), respectively, of a nonspecifically-designed AR system that is inthe form of a smartphone 10 and that can be configured or designed toimplement one or more AR functionalities. FIG. 1B shows the backside 14(e.g., the side of a personal computing/communication device that isopposite of the front-side 12 personal computing/communication device)of the exemplary smartphone 10 of FIG. 1A. The exemplary smartphone 10may include a display 18, such as a touchscreen or liquid crystaldisplay (LCD), on the front-side 12 of the smartphone 10. The smartphone10 of FIGS. 1A and 1B is also depicted as having lens 16 a for aninward-facing camera on the front-side 12 and lens 16 b for aforward-facing camera on the back-side 14 of the smartphone 10. Whenemployed to provide one or more augmented views of one or more scenes ofreal world environments, the display 18 may be used in order to displaythe augmented views. In some cases, the inward-facing camera associatedwith lens 16 a may be used in order to track the gaze, focus, dwellpath, and/or dwell time of one or more eyes of a user. Theforward-facing camera that is associated with lens 16 b, on the otherhand, may be used to capture (e.g., sense and/or record) actual scenesof real world environments in order to generate augmented views of thoseactual scenes. The generated augmented views may then be displayedthrough display 18.

FIGS. 2A and 2B illustrates the front-side 22 (e.g., the side of apersonal computing/communication device that a display is located ordisposed on) and the backside 24 (e.g., the side of thecomputing/communication device that is opposite of the front-side of thecomputing/communication device), respectively, of anonspecifically-designed AR system that is in the form of a tabletcomputer 20 and that can be configured or designed to implement one ormore AR functionalities. In FIGS. 2A and 2B, the tablet computer 20 isdepicted as having a display 28, such as a touchscreen, on thefront-side 22 of the tablet computer 20. The tablet computer 20 is alsodepicted as having lens 26 a for an inward-facing camera on thefront-side 22 and lens 26 b for a forward-facing camera on the back-side24 of the tablet computer 20.

In some cases, the inward-facing camera associated with lens 26 a may beused in order to track the gaze, focus, dwell path, and/or dwell time ofone or more eyes of a user. The forward-facing camera associated withlens 26 b on the back-side 24 of the tablet computer 20 may be used tocapture (e.g., sense and/or record) actual scenes of real worldenvironments in order to generate augmented views of those actualscenes. Note that in some cases, a single camera may be coupled to bothlens 26 a and lens 26 b, thus the single camera can act as both aforward-facing and rear-facing camera. The generated augmented views maythen be displayed through display 28. References to “real environment”or “real world environment” herein may be in reference to true or actualphysical environments rather than to a virtual environment or virtualworld.

FIGS. 3A and 3B illustrate a specifically-designed AR system in the formof video goggles 30 that can implement one or more AR functionalities.In particular, FIG. 3A illustrates a perspective view of the videogoggle 30, which includes, among other things, a frame piece 32, a lefttemple 31 a, and a right temple 31 b. Disposed on the front-side 39 a ofthe frame piece 32 are two forward looking lenses 34 a and 34 b for apair of corresponding forward-looking cameras for capturing (e.g.,recording, scanning, sensing, etc.) actual scenes of real worldenvironments in order to generate augmented views of those actualscenes. FIG. 3B illustrates a view of the backside 39 b of the framepiece 32 of the exemplary video goggles 30 of FIG. 3A. The backside 39 bof the frame piece 32 is depicted as including a left display 38 a and aright display 38 b, a rim 35 surrounding the left display 38 a and theright display 38 b, a left lens 36 a for a left inward-facing camera anda right lens 36 b for a right inward-facing camera, and a left hinge 33b and a right hinge 33 a. The rim 35 that may surround the left display38 a and the right display 38 b may be a soft, semi-soft, or hard rimthat in some cases may act as a cushion, as well as a barrier to preventexcess light from entering a user's field of view (thus allowing theuser to better view the left and right displays 38 a and 38 b).

The forward-looking cameras associated with the forward-looking lens 34a and 34 b may be used to capture (e.g., sense and/or record) actualscenes of real world environments in order to generate augmented viewsof those actual scenes. These augmented views that are generated maythen be displayed through displays 38 a and 38 b. Note that inalternative implementations, the video goggle 30 may only employ asingle forward-looking lens for a single forward-looking camera ratherthan employing two forward-looking lenses 34 a and 34 b for twoforward-looking cameras as illustrated in FIG. 3A. In suchimplementations, the single forward-looking lens may be disposed at themid-point or bridge part of the frame piece 32 above where a user's nosewould be positioned. The output associated with the corresponding singleforward-looking camera would then be displayed through both displays 38a and 38 b.

The video goggles 30 may also include inward-looking lenses 36 a and 36b as depicted in FIG. 3B for two inward-looking cameras (not shown) onthe backside 39 b of the frame piece 32 and disposed between thedisplays 38 a and 38 b and rim 35. The inward-looking cameras may beemployed in order to track the movements as well as the gaze, focus,dwell path, and/or dwell time of one or more eyes of a user. Note thatalternatively, the video goggles 30 may include fewer or moreinward-looking cameras and inward-looking lenses. Further, there is norequirement that a corresponding camera is needed for eachinward-looking lens as it may be possible to employ, for example, asingle camera for viewing through multiple lenses.

The frame piece 32, the left temple 31 a, and the right temple 31 b mayhouse various electronics that are designed for, for example, processingdata provided by the various cameras (e.g., forward-looking as well asbackward looking cameras), and for generating augmented views of scenesfrom real world environment that may be displayed through displays 38 aand 38 b. The types of electronics that may be included with the videogoggles 30 may include, for example, the same or similar types ofelectronics (e.g., microprocessors, controllers, network interface card,memory, etc.) that are often found in mobile computing/communicationdevices such as the smartphone 10 or the tablet computer 20 describedearlier. The left temple 31 a and the right temple 31 b are featuresthat allow the AR system to be worn on a user's head.

Turning now to FIGS. 4A and 4B, which illustrate a specifically-designedAR system that is in the form of electronic glasses 40 that canimplement one or more AR functionalities. In particular, FIG. 4Aillustrates a perspective view of the electronic glasses 40, whichincludes, among other things, a frame piece 41 (which further includes aleft rim piece 43 a, a right rim piece 43 b, and a bridge 44), a lefttemple 42 a, a right temple 42 b, a left lens 45 a, a right lens 45 b, asee-through display 46, and electronics housing 47 (note that in somecases, the electronics housing 47 may include an optional clip 47 a forcoupling the electronics housing 47 to the right temple 42 b. The framepiece 41 having a front-side 49 a and a backside 49 b opposite of thefront-side 49 a. Disposed at the end of the electronics housing 47 is aforward-looking lens 48 a for a corresponding forward-looking camera forcapturing (e.g., recording, scanning, sensing, etc.) actual scenes ofreal world environments in order to generate augmented views of thoseactual scenes. In some alternative implementations, the forward-lookinglens 48 a may be alternatively located at bridge 44 of the frame piece41 or at some other location. Note that the left lens 45 a and the rightlens 45 b are optional and are not necessary for implementing ARfunctionalities.

In FIG. 4A, the see-through display 46 is depicted as covering only aportion of the right lens 45 b and being depicted as being attached tothe end of the electronics housing 47. Thus, the see-through display 46may be used to display and overlay computer generated data and/or imagesonto portions of views of actual scenes of the real world environmentthat a user might see through right lens 45 b. Note again that since thesee-through display 46 covers only a portion of the right lens 45 b,only a portion of the view that a user may see through the right lens 45b may be augmented (e.g., modified). In some other alternativeimplementations, the see-through display 46 may alternatively cover theentire right lens 45 b so that the entire view of the user through theright lens 45 b may be augmented if needed. Although the electronicglasses 40 in FIGS. 4A and 4B is depicted as having only one see-throughdisplay 46 over the right lens 45 b, in alternative implementations, asecond see-through display may be disposed over the left lens 45 a. Theleft temple 42 a and the right temple 42 b are features that allow theAR system to be worn on a user's head.

FIG. 4B depicts a view of the backside 49 b of the frame piece 41 of theelectronic glasses 40 depicted in FIG. 4A. In FIG. 4B, the see-throughdisplay 46 can be seen through the clear right lens 45 b. Furtherillustrated in FIG. 4B is an inward-looking lens 48 b that can be seenthrough the clear right lens 45 b and which is for a correspondinginward-looking camera. As illustrated the inward-looking lens 48 b isdisposed on the electronics housing 47 near the see-through display 46.The inward-looking cameras may be employed in order to track themovements as well as the gaze, focus, dwell path, and/or dwell time ofthe right eye of a user. The placement of the inward-looking lens 48 bis a design choice and may be located elsewhere so long as it has a viewto the right eye of a user. In alternative implementations, a secondinward-looking lens for a second inward-looking camera may be includedin order to track and monitor the movements as well as the gaze, focus,dwell path, and/or dwell times of the left eye of the user. In theexemplary electronic glasses 40 illustrated in FIG. 4A, all of the majorcomponents are depicted as being located on the right-side of theelectronic glasses 40. Alternatively, these components (e.g., electronichousing 47, see-through display 46, and so forth) may be alternativelyor additionally located on the left-side of the electronic glasses 40.

The electronics housing 47 may house various electronics includingelectronics that are designed for, for example, processing data providedby the various cameras (e.g., forward-looking as well as inward lookingcameras), and for generating augmented views of scenes from real worldenvironment that may be displayed through see-through display 46. Thetypes of electronics that may be included with the electronic glasses 40may include, for example, the types of electronics (e.g.,microprocessors, controllers, network interface card, memory, camera,battery, etc.) that are often found in mobile computing/communicationdevices such as the smartphone 10 or the tablet computer 20 describedearlier.

In some cases, the electronic housing 47 (and its contents including oneor more cameras) and the see-through display 46 may be a separate unitthat can be clipped onto a prescription or non-prescription eyeglasses.In such an embodiment, the electronic housing 47 may include one or morefeatures (e.g. one or more clips, magnets, straps, and so forth) thatallows the housing to be worn by a user by allowing the electronichousing 47 to be attached to a pair of eye-glasses or goggles. Note thatalthough not depicted herein, AR devices may come in other types offorms other than those illustrated in FIGS. 1A, 1B, 2A, 2B, 3A, 3B, 4A,and 4B including, for example, a helmet, a mask, and so forth.

There are many types of augmentations that can be provided through ARsystems including, for example, augmentations in the form of text thatmay be added to an augmented view, an augmentation in the form of 2 or3-dimensional visual item (which may or may not be an animated item thatmoves) that may be added to an augmented view, and/or an augmentationthat simply modifies, emphasizes, deletes, or de-emphasizes an existingreal world item (e.g., intensity of light, color of a car, removal ofundesirable elements in the corresponding real field of view such asrubbish on the ground, etc.) in the augmented view.

One type of augmentation that many currently available AR systems caninclude into an augmented view of an actual scene of the realenvironment are vision-based augmentations that depend, for theirexistence in the augmented view, on the presence of a visual cue (e.g.,an “anchor” visual item) in the actual view of a scene from the realenvironment. That is, in many currently available AR systems, anaugmentation will only be included into an augmented view only when avisual cue is detected in the actual view of a scene from the realenvironment. In some cases, the visual cue that may be the basis for theinclusion of the augmentation in the augmented view may be a visualmarker (e.g., a recognizable symbol or pattern such as the face of adollar bill) that may have been purposefully placed in an actual sceneof the real environment in order to prompt an AR system to insert aparticular augmentation into the augmented view of the actual scene ofthe real environment. In other cases, the visual cue may be arecognizable or identifiable visual pattern (e.g., a human face) thatmay be recognized using image feature analysis (e.g., image trackingsuch as Parallel Tracking).

There are, of course, other types of augmentations that currentlyavailable systems can provide or have been proposed that depend, fortheir existence in an augmented view, on factors other than visual cuesin the actual views of scenes of the real environment. For example, insome current AR systems, an augmentation can be provided that maydepend, for its existence in the augmented view, on the location and/ororientation of the AR system that is presenting (e.g., displaying and/ortransmitting) the augmented view. In other cases, an AR system may beconfigured or designed to insert one or more augmentations into anaugmented view based on the location of the AR system and/or whichdirection is the AR system “facing” (e.g., east, west, north, up, down,etc.).

It is contemplated that as the AR technology evolves, one of thechallenges that AR developers may face is determining and applying theoptimal formatting (e.g., size, shape, color, length of visibility,etc.), inclusion (e.g., when to include into an augmented view), andplacement (e.g., placement in the augmented view) of augmentations. Thatis, it is likely that with respect to at least some types ofaugmentations it may be highly desirable for such augmentations to havecertain formats and/or be placed at certain locations of an augmentedview in order to maximize their visibility or “noticeability.” It isalso just as likely that with respect to other types of augmentations itmay be highly desirable for such augmentations to have particularformats and/or be placed at particular locations of an augmented view inorder to minimize or reduce their visibility or noticeability. Theoptimal formatting and/or placement of augmentations in augmented viewsmay, therefore, be particularly useful for presenting augmented views ofreal scenes from the real environment.

In accordance with various embodiments, computationally implementedmethods, systems, circuitry, articles of manufacture, and computerprogram products are described herein that are designed to, among otherthings, ascertain one or more user reactions to a display of anaugmented view of an actual scene from a real environment, the displayedaugmented view having one or more augmentations; and correlating thedetected one or more user reactions with at least one or more aspectsassociated with the one or more augmentations. In various embodiments,the results of the correlation may be particularly useful for, amongother things, determining when and how to present a particularaugmentation in an augmented view in order to, for example, maximize orminimize its visibility or noticeability.

More particularly, the computationally implemented methods, systems,circuitry, articles of manufacture, and computer program products may bedesigned to, among other things, detecting one or more user reactions ofa user in response to a display to the user of an augmented view of anactual scene from a real environment, the augmented view that wasdisplayed including one or more augmentations, and correlating thedetected one or more user reactions with at least one or more aspectsassociated with the one or more augmentations that were included in theaugmented view that was presented. In some embodiments, thecomputationally implemented methods, systems, circuitry, articles ofmanufacture, and computer program products may be further designed totransmitting one or more results of the correlating; and receiving, inresponse to said transmitting, one or more second augmentations fordisplaying in a second augmented view of the actual scene or of a secondactual scene from the real environment. In the same or differentembodiments, the computationally implemented methods, systems,circuitry, articles of manufacture, and computer program products mayadditionally or alternatively be designed to displaying a secondaugmented view of the actual scene or of a second actual scene from thereal environment, the second augmented view including one or more secondaugmentations that have been included into the second augmented viewbased, at least in part, on the correlating.

In various embodiments, the methods, systems, circuitry, articles ofmanufacture, and computer program products in accordance with variousembodiments may be implemented by the AR device 70* of FIG. 7A or 7B.Note that for purposes of the following, “*” represents a wildcard.Thus, references in the following to the AR device 70* of FIG. 7A or 7Bmay be in reference to the AR device 70′ of FIG. 7A or to the AR device70″ of FIG. 7B. Note further that the AR device 70* of FIG. 7A or 7B mayhave a variety of form factors including any one of the form factorsillustrated in FIGS. 1A, 1B, 2A, 2B, 3A, 3B, 4A, and 4B, or having otherform factors (e.g., a helmet, a mask, and so forth).

In order to further facilitate understanding of the various operations,processes, and systems to be described herein, certain concepts andterms will now be introduced with references to FIGS. 5A and 5B. Inparticular, FIG. 5A is a top-down view of an AR user 52 (herein simply“user 52”) using an AR device 70* that is in the form of a tabletcomputer 20 (as illustrated in FIGS. 2A and 2B) at different points orincrements in time. Note that although the AR device illustrated here isin the form of a tablet computer 20, the concepts to be introduced hereas well as in FIG. 5B will apply to other AR devices having other formfactors (e.g., a smartphone, a video goggle, electronic glasses, and soforth).

Referring back to FIG. 5A which shows user 52 viewing actual views 51 ofactual scenes from the real (physical) environment 50 surrounding the ARdevice (e.g., tablet computer 20). The actual views 51 of the actualscenes of the real environment 50 may be captured by a camera throughlens 26 b at different points in time. That is, an actual scene from thereal environment 50 is not only location specific but is also timespecific. In this example, the actual views 51 captured by the camera ofthe tablet computer 20 is the view of a store front. Note that a car isillustrated as being present in the real environment 50 and is depictedas moving towards the line of sight (e.g., actual view 51—field of view)of the tablet computer 20. Thus, although the field of view for thecamera of the tablet computer 20 is directed to the same location overlength of time, the actual views 51 of the location will change overtime (e.g., the actual views 51 will change as, for example, the car (orpedestrians) comes into and/or leaves the field of view of the tabletcomputer 20.

After capturing the actual views 51 through lens 26 b, the tabletcomputer 20 may display through a display 28 one or more augmented views56 (each of which include one or more augmentations). The user 52 mayview the one or more augmented views 56 visually presented through thedisplay 28 using his or her eyes 502. The inward-facing camera throughlens 26 a may track the eyes 502 of the user 52 as the user 52 views theone or more augmented views 56. In particular, the tablet computer 20through its inward facing camera may track one or more characteristicsof the eyes 502 of the user 52, including, for example, eye focus 502,dwell path, dwell time, pupil size and shape, and so forth. In variousembodiments, a dwell path is the focus path of one or more eyes 502 of auser 52 as the user 52, views, for example, actual and/or augmentedviews that may be displayed through an AR device (e.g., tablet computer20). The dwell path may be defined or identified by tracking the paththat focus point of the one or more eyes 502 takes along, for example,an augmented view 56 (or non-augmented or actual view) displayed throughthe AR device (e.g., tablet computer 20). In contrast, dwell time is theamount of time that a user dwells on (e.g., stares at or focuses on) a,for example, visual object.

Referring now to 5B, which illustrates a side-view of the user 52 usingthe AR device (e.g., tablet computer 20) as illustrated in FIG. 5A. Thedisplay 28 may visually present one or more augmented views 56 based, atleast in part, on one or more captured actual views 51. Theinward-facing camera of the tablet computer 20 and through lens 26 a maytrack the one or more eyes 504 of the user 52 including, for example,the eye focus 502 of the user 52 as well as the dwell path and/or dwelltime of the user 52 in response to the display of one or more augmentedviews 56.

Turning now to FIGS. 6A, 6B, 6C, 6D, 6E, 6F, 6G, 6H, 6I, 6K, and 6M,that illustrate exemplary actual views and augmented views of variousscenes from the real environment that may be provided by thecomputationally implemented methods, systems, circuitry, articles ofmanufacture, and computer program products in accordance with variousembodiments. The exemplary actual views and the exemplary augmentedviews illustrated in these figures are provided in order to facilitateunderstanding of various concepts that will be described in connectionwith the various operations and processes to be discussed herein. Notethat in various embodiments at least the augmented views (e.g.,augmented views 60 b, 60 c, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k and 60 m)illustrated in FIGS. 6B, 6C, 6D, 6E, 6F, 6G, 6I, 6K and 6M may bepresented by the AR device 70* (e.g., AR device 70′ or AR device 70″) ofFIG. 7A or 7B.

Referring particularly now to FIG. 6A, which illustrates an actual view60 a of a scene from the real environment. In this case, the actualscene 60 a is scene of a beach. Based on the actual view 60 a, the ARdevice 70* of FIG. 7A or 7B may generate an augmented view 60 b asillustrated in FIG. 6B. The augmented view 60 b includes an augmentation64 a (configured to be in the shape of a sundial) that may have beenincluded in the augmented view 60 b based on one or more visibilityrules that the AR device 70* may be executing. Visibility rules aredirectives or instructions that define when or how an augmentation maybe presented. For example, in some cases, a visibility rule may requirea specific augmentation to be included into an augmented view when aparticular “anchor” visual cue is detected as being present in thecorresponding actual scene of the real environment that the augmentedview is based on. Other visibility rules may direct the inclusion of anaugmentation into an augmented view based on the location and/ororientation of the AR device 70*.

FIG. 6C illustrates an augmented view 60 c of the actual scene (e.g.,actual view 60 a) depicted in FIG. 6A. Note that augmented view 60 c isessentially the same as the augmented view 60 b of FIG. 6B except that adwell path 66 a of one or more eyes of a user 52 in response to andduring and/or following the display of the augmented view 60 c has beenoverlaid on top of the augmented view 60 c. Also illustrated in FIG. 6Cis the focus point of the one or more eyes of the user 52, which willnow be referred to as eye focus (e.g., eye focus 65 a in FIG. 6C)

Based on the detected reaction (e.g., dwell path 66 a as illustrated inFIG. 6C) of a user 52 to the display of the augmented view 60 b (seeFIG. 6B) including the augmentation 64 a, the AR device 70* may visuallypresent the augmented view 60 d of FIG. 6D. The augmented view 60 dincludes an augmentation 64 b having a particular format (e.g., a shapeof a soft drink bottle) and being placed at a particular location (e.g.,right bottom corner of the augmented view 60 d and on top of the beach)in the augmented view 60 d. The format and/or placement of theaugmentation 64 b may be based on the detected reaction of the user(e.g., eye focus and dwell path, perspiration level of the user 52 basedon detected skin characteristics, etc.).

FIG. 6E illustrates another example augmented view 60 e of the beachscene illustrated in FIG. 6A. The example augmented view 60 e includesan augmentation 64 c that is an informational augmentation indicatingenvironmental information (that shows 90 percent chance ofprecipitation). Illustrated also in FIG. 6E is the dwell path 66 b andthe eye focus 65 b (e.g., eye focus point) of the user 52 during and/orfollowing the display of the augmented view 60 e. The example dwell path66 b of the user 52 that is directed skywards in FIG. 6E may have beenas result of the user seeing that the augmentation 64 c indicating a 90percent chance of precipitation.

FIG. 6F illustrates another example augmented view 60 f that may begenerated by the AR device 70* of FIG. 7A or 7B. The augmented view 60 fmay have been generated based on the detected reaction (e.g., dwell path66 b of FIG. 6E) of the user 52 to the display of the augmented view 60e of FIG. 6E. Note that augmented view 60 f may be a different augmentedview from the augmented view 60 e of FIG. 6E (e.g., augmented view 60 emay have presented on Monday while augmented view 60 f may have beenpresented two days later on Wednesday looking at the same location butwith similar conditions). Augmented view 60 f includes augmentation 64cc (which includes similar but not the same environmental information asaugmentation 64 c of FIG. 6E) and augmentation 64 d (which may have beenformatted and selectively placed in the top right corner of theaugmented view 60 f based on the reaction of the user to the augmentedview 60 e of FIG. 6E). The augmentation 64 d being in the shape of aplane towing a banner. Augmentation 64 cc includes an augmentationaspect 67 a in the form of a darkened or darkly colored boarder. FIG. 6Falso illustrates eye focus 65 c of the one or more eyes of the user 52being on the augmentation aspect 67 a.

Referring now to FIG. 6G illustrates another example augmented view 60 gof the beach scene illustrated in FIG. 6A. In particular, FIG. 6Gillustrates the dwell path 66 b of one or more eyes of a user 52 betweenaugmentation 64 c and augmentation 64 e. The roundabout path of thedwell path 66 b may be as result of the precipitation informationincluded in the augmentation 64 c. The detected dwell path 66 b may, insome embodiments, cause the AR device 70* to selectively insert anyadditional augmentations along (or not along) the detected dwell path 66b.

FIG. 6H illustrates an example actual view 60 h of an interior scenefrom a shopping mall as captured by, for example, the AR device 70* ofFIG. 7A or 7B. Referring now to FIG. 6I, which illustrates an augmentedview 60 i of the actual view 60 h illustrated in FIG. 6H. Included inthe augmented view is an augmentation 64 f, which is an informationalaugmentation that provides menu information of a restaurant that islocated in the shopping mall.

Turning now to FIG. 6K, which illustrates the augmented view 60 k of theinterior of the restaurant that was referred to above. Included in theaugmented view 60 k is an augmentation 64 g, which contains the sameinformation as the augmentation 64 f of FIG. 6I. Note, however, theaugmentation 64 g has been placed at the left bottom corner of theaugmented view 60 k rather than in the center of the augmented view 60 kas was the case in the augmented view 60 i of FIG. 6I. In some cases,the placement of the augmentation 64 g at the corner of the augmentedview 60 k may be based on the user 52 reaction to the augmented view 60i of FIG. 6I. That is, the AR device 70* may have detected that the user52 showed very little interest (e.g., very low dwell time or lack of eyefocus) in augmentation 64 f because the user may have been, for example,irritated by the placement of the augmentation 64 f at the center of theaugmented view 60 i. As a result, the augmentation 64 g was selectivelyplaced at the corner of the augmented view 60 k in order to notantagonize the user 52.

FIG. 6M shows another example augmented view 60 m of the interior of ashopping mall as was illustrated in FIG. 6H. The augmented view 60 mincludes an augmentation 64 h that contains similar information asaugmentation 64 f of FIG. 6I. Note, however, augmentation 64 h has adifferent format (e.g., different coloring and a bolder border) from theformat of augmentation 64 f. In various embodiments, the augmentation 64h may be as a result of the AR device 70* detecting user 52 lack ofinterest of the original augmentation 64 f of FIG. 6I. Note that a moredetailed discussion related to FIGS. 6A, 6B, 6C, 6D, 6E, 6F, 6G, 6H, 6I,6K and 6M will be provided with respect to the operations and processesto be described herein.

Referring now to FIGS. 7A and 7B, illustrating two block diagramsrepresenting two different implementations of an augmented reality (AR)device 70* that can execute the operations and processes to be describedherein. In particular, and as will be further described herein, FIG. 7Aillustrates an AR device 70′ that is the “hardwired” or “hard”implementation of an AR device 70′ that can implement the operations andprocesses to be described herein. The AR device 70′ may include certainlogic modules including, for example, a user reaction ascertainingmodule 102′, a user reaction associating module 104′, a non-augmentedview presenting module 105′, an augmented view presenting module 106′, acorrelation result relaying module 107′, and/or an augmentationacquiring module 108′ (which may further include a visibility ruleacquiring module 109′) that are implemented using purely hardware orcircuitry components (e.g., application specific integrated circuit (orASIC). In contrast, FIG. 7B illustrates an AR device 70″ that is the“soft” implementation of the AR device 70′ of FIG. 7A in which certainlogic modules including a user reaction ascertaining module 102″, a userreaction associating module 104″, a non-augmented view presenting module105″, an augmented view presenting module 106″, a correlation resultrelaying module 107″, and an augmentation acquiring module 108″ (whichmay further include a visibility rule acquiring module 109″) areimplemented using electronic circuitry (e.g., one or more processorsincluding one or more microprocessors, controllers, etc.) executing oneor more programming instructions (e.g., software).

The embodiments of the AR device 70* illustrated in FIGS. 7A and 7B aretwo extreme implementations of the AR device 70* in which all of thelogic modules (e.g., the user reaction ascertaining module 102′, theuser reaction associating module 104′, the non-augmented view presentingmodule 105′, the augmented view presenting module 106′, the correlationresult relaying module 107′, and the augmentation acquiring module 108′including the visibility rule acquiring module 109′) are implementedusing purely hardware solutions (e.g., circuitry such as ASIC) asillustrated in FIG. 7A or in which all of the logic modules (e.g., theuser reaction ascertaining module 102″, the user reaction associatingmodule 104″, the non-augmented view presenting module 105″, theaugmented view presenting module 106″, the correlation result relayingmodule 107″, and the augmentation acquiring module 108″ including thevisibility rule acquiring module 109″) are implemented using softwaresolutions (e.g., programmable instructions being executed by hardwaresuch as one or more processors) as illustrated in FIG. 7B. Since, thereare many ways of combining hardware, software, and/or firmware in orderto implement the various logic modules (e.g., the user reactionascertaining module 102*, the user reaction associating module 104*, thenon-augmented view presenting module 105*, the augmented view presentingmodule 106*, the correlation result relaying module 107*, and theaugmentation acquiring module 108* including the visibility ruleacquiring module 109*), only the two extreme implementations (e.g., thepurely hardware solution as illustrated in FIG. 7A and the softwaresolution of FIG. 7B) are illustrated here. It should be noted here thatwith respect to the “soft” implementation illustrated in FIG. 7B,hardware in the form of circuitry such as one or more processors 116 arestill needed in order to execute the software. Further details relatedto the two implementations of AR device 70* illustrated in FIGS. 7A and7B will be provided in greater detail below.

In some instances, one or more components may be referred to herein as“configured to,” “configured by,” “configurable to,” “operable/operativeto,” “adapted/adaptable,” “able to,” “conformable/conformed to,”“designed to,” etc. Those skilled in the art will recognize that suchterms (e.g., “configured to”) generally encompass active-statecomponents and/or inactive-state components and/or standby-statecomponents, unless context requires otherwise.

Referring particularly now to FIG. 7A, which illustrates a block diagramof an AR device 70′ that includes a user reaction ascertaining module102′, a user reaction associating module 104′, a non-augmented viewpresenting module 105′, an augmented view presenting module 106′, acorrelation result relaying module 107′, and an augmentation acquiringmodule 108′ (which further includes a visibility rule acquiring module109′), one or more processors 116 (e.g., one or more microprocessors), amemory 114 that may store one or more applications 160 (e.g., anoperating system (OS) 162, one or more facial recognition applications163, one or more visual object recognition applications 164 (e.g., forvisually recognizing one or more items that are visible in a scene fromthe real physical environment), one or more voice recognitionapplications 165, one or more sound recognition applications 166 (e.g.,an application for recognizing/identifying sounds other than voice suchas the sound of a train passing or waves crashing along a shoreline),and/or one or more personal information manager (PIM) applications 168),one or more augmentations 170, and one or more visibility rules 171(e.g., directives for when and how augmentations are to be presented), auser interface 110 (e.g., a display, a mouse, a microphone, etc.), anetwork interface 112 (e.g., a network interface card or “NIC”), and oneor more sensors 120. Although not depicted, the memory 114 may furtherinclude other types of application not depicted in FIG. 7A including,for example, communication applications such as an email applicationand/or an instant messaging (IM) application.

In various embodiments, the user reaction ascertaining module 102′ ofFIG. 7A is a logic module that may be designed to, among other things,ascertain one or more user reactions of a user reacting to a display ofan augmented view of an actual scene from a real environment, thedisplayed augmented view having included one or more augmentations 170.

In contrast, the user reaction associating module 104′ of FIG. 7A is alogic module that may be configured to associate the one or more userreactions with at least one or more aspects associated with the one ormore augmentations 170 that were displayed with the augmented view.

Turning now to FIG. 7B, which illustrates a block diagram of another ARdevice 70″ that can implement the operations and processes to bedescribed herein. As indicated earlier, the AR device 70″ in FIG. 7B ismerely the “soft” version of the AR device 70′ of FIG. 7A because thevarious logic modules: the user reaction ascertaining module 102″, theuser reaction associating module 104″, the non-augmented view presentingmodule 105″, the augmented view presenting module 106″, the correlationresult relaying module 107″, and the augmentation acquiring module 108″including the visibility rule acquiring module 109″ are implementedusing software and one or more processors 116 (e.g., one or moremicroprocessors or controllers) executing the software (e.g., computerreadable instructions 152) rather than being implemented using purelyhardware (e.g., ASIC) as was the case in the AR device 70′ of FIG. 7A.Thus, the user reaction ascertaining module 102″, the user reactionassociating module 104″, the non-augmented view presenting module 105″,the augmented view presenting module 106″, the correlation resultrelaying module 107″, and the augmentation acquiring module 108″including the visibility rule acquiring module 109″ of FIG. 7B may bedesigned to execute the same functions as the user reaction ascertainingmodule 102′, the user reaction associating module 104′, thenon-augmented view presenting module 105′, the augmented view presentingmodule 106′, the correlation result relaying module 107′, and theaugmentation acquiring module 108′ including the visibility ruleacquiring module 109′ of FIG. 7A. The AR device 70″, as illustrated inFIG. 7B, has other components (e.g., user interface 110, networkinterface 112, and so forth) that are the same or similar to the othercomponents included in the AR device 70′ of FIG. 7A. Note that in theembodiment of the AR device 70* illustrated in FIG. 7B, the variouslogic modules (e.g., the user reaction ascertaining module 102″, theuser reaction associating module 104″, and so forth) may be implementedby the one or more processors 116 (or other types of circuitry such asfield programmable gate arrays or FPGAs) executing one or more computerreadable instructions 152 stored in memory 114.

In various embodiments, the memory 114 of the AR device 70′ of FIG. 7Aand the AR device 70″ of FIG. 7B may comprise of one or more of massstorage device, read-only memory (ROM), programmable read-only memory(PROM), erasable programmable read-only memory (EPROM), cache memorysuch as random access memory (RAM), flash memory, synchronous randomaccess memory (SRAM), dynamic random access memory (DRAM), and/or othertypes of memory devices.

Turning now to FIG. 8A illustrating a particular implementation of theuser reaction ascertaining module 102* (e.g., the user reactionascertaining module 102′ or the user reaction ascertaining module 102″)of FIGS. 7A and 7B. As illustrated, the user reaction ascertainingmodule 102* may include one or more sub-logic modules in variousalternative implementations. For example, in various embodiments, theuser reaction ascertaining module 102* may include a physiologicalcharacteristic determining module 802 that includes an eyecharacteristic determining module 804, which may further include an eyemovement determining module 806 (which may further include an eye dwellpath detecting module 808), an eye attention determining module 810(which may further include an eye attention time determining module812), and/or a pupil characteristic detecting module 814, and/or anaugmented view aspect recording module 816 that includes an augmentationform recording module 818 (which may further include an augmentationshape/dimension recording module 820 and/or an augmentation locationrecording module 822), a non-augmentation scene element aspect recordingmodule 824 (which may further include a non-augmentation scene elementlocation recording module 826), and/or an obfuscated non-augmentationscene element recording module 828. Specific details related to the userreaction ascertaining module 102* as well as the above-describedsub-modules of the user reaction ascertaining module 102* will beprovided below with respect to the operations and processes to bedescribed herein.

FIG. 8B illustrates a particular implementation of the user reactionassociating module 104* (e.g., the particular implementation of the userreaction associating module 104′ or the particular implementation of theuser reaction associating module 104″) of FIG. 7A or 7B. As illustrated,the user reaction associating module 104* may include one or moresub-logic modules in various alternative embodiments. For example, invarious embodiments, the user reaction associating module 104* mayinclude a relationship identifying module 830 (which may further includea relationship logging module 832) and/or a physiological characteristicassociating module 834 that may further include an eye characteristicassociating module 836, which may further include an eye movementassociating module 838 (which may further include a dwell pathassociating module 840), an eye focus associating module 842 (which mayfurther include a dwell time associating module 844), and/or a pupilcharacteristic associating module 846 (which may further include a pupilshape/size associating module 848). Specific details related to the userreaction associating module 104* as well as the above-describedsub-modules of the user reaction associating module 104* will beprovided below with respect to the operations and processes to bedescribed herein.

FIG. 8C illustrates the various types of user interface devices that maybe part of the user interface 110 of the AR device 70* of FIG. 7A or 7B.In various embodiments, the user interface 110 may include one or moredisplay monitors 852 (e.g., a touchscreen, a liquid crystal display(LCD), a see-through display, and/or other types of display monitors),one or more visual capturing devices 854 (e.g., one or more video or webcameras, digital cameras, and/or other types of cameras 870—see FIG.8D), one or more audio speakers 856, one or more audio input devices 858(e.g., one or more microphones)—see also audio sensors 872 of FIG. 8D,and/or one or more keyboard/keypads 860. Although not depicted, othertypes of user interfaces 110 may be included with the user interface 110in various alternative embodiments including, for example, a mouse orother types of user input/output devices.

FIG. 8D illustrates at least some of the various types of sensors 120that may be included with the AR device 70* (e.g. the AR device 70′ ofFIG. 7A or the AR device 70″ of FIG. 7B). As illustrated, the one ormore sensors 120 that may be included with the AR device 70* may includeone or more cameras 870 (note that the one or more cameras 870 may bethe same as the one or more visual capturing devices 854 described abovewith respect to the user interface 110), one or more audio sensors 872(see also audio input device[s] 858 described above), one or morelocation sensors 874 such as one or more global positioning systems(GPSs) 875, one or more orientation/movement sensors 876 (which maycomprise one or more accelerometers 877, one or more gyroscopes 878, oneor more inertia sensors 879, one or more pedometers 880), and/or one ormore user physiological sensors 882 (e.g., one or more blood pressuresensors 883, one or more pulse/heart rate sensors 884, one or moregalvanic skin sensors 885). Note that with respect to the one or morelocation sensors 874, the one or more orientation/movement sensors 876,and/or one or more user physiological sensors 882, these sensor devicesmay include other types of sensors not depicted in FIG. 8D. For examplethe one or more location sensors 874 in some alternative implementationsmay include a location sensor 874 that employs triangulation techniquesand signals from cellular towers to determine location while the one ormore user physiological sensors 882 may include in some alternativeembodiments one or more blood pressure sensors 883 and/or other types ofsensors 120.

A more detailed discussion related to the AR device 70* (e.g., the ARdevice 70′ of FIG. 7A or the AR device 70″ of FIG. 7B) discussed abovewill now be provided with respect to the processes and operations to bedescribed herein. FIG. 9 illustrates an operational flow 900representing example operations for, among other things, ascertaining auser reaction to a display of an augmented view of an actual scene fromthe real physical environment, the augmented view to be displayedincluding one or more augmentations; and associating the user reactionto one or more aspects of the one or more augmentations. In variousimplementations, the results of these operations (e.g., association orcorrelation of the user reaction with the one or more aspects of the oneor more augmentations) may be used in order to, for example, optimallyformat augmentations that may be displayed in subsequent augmentedviews.

In FIG. 9 and in the following figures that include various examples ofoperational flows, discussions and explanations will be provided withrespect to the AR device 70* described above and as illustrated in FIGS.1A, 1B, 2A, 2B, 3A, 3B, 4A, 4B, 7A, 7B, 8A, 8B, 8C, and 8D, and/or withrespect to other examples (e.g., as provided in FIGS. 5A, 5B, 6A, 6B,6C, 6D, 6E, 6F, 6G, 6H, 6I, 6K, and 6M) and contexts. However, it shouldbe understood that the operational flows may be executed in a number ofother environments and contexts, and/or in modified versions of FIGS.1A, 1B, 2A, 2B, 3A, 3B, 4A, 4B, 5A, 5B, 6A, 6B, 6C, 6D, 6E, 6F, 6G, 6H,6I, 6K, 6M, 7A, 7B, 8A, 8B, 8C, and 8D. Also, although the variousoperational flows are presented in the sequence(s) illustrated, itshould be understood that the various operations may be performed inother orders other than those which are illustrated, or may be performedconcurrently.

Further, in FIG. 9 and in the figures to follow thereafter, variousoperations may be depicted in a box-within-a-box manner. Such depictionsmay indicate that an operation in an internal box may comprise anoptional example embodiment of the operational step illustrated in oneor more external boxes. However, it should be understood that internalbox operations may be viewed as independent operations separate from anyassociated external boxes and may be performed in any sequence withrespect to all other illustrated operations, or may be performedconcurrently. Still further, these operations illustrated in FIG. 9 aswell as the other operations to be described herein are performed by atleast one of a machine, an article of manufacture, or a composition ofmatter unless indicated otherwise.

For ease of understanding, the flowcharts are organized such that theinitial flowcharts present implementations via an example implementationand thereafter the following flowcharts present alternateimplementations and/or expansions of the initial flowchart(s) as eithersub-component operations or additional component operations building onone or more earlier-presented flowcharts. Those having skill in the artwill appreciate that the style of presentation utilized herein (e.g.,beginning with a presentation of a flowchart(s) presenting an exampleimplementation and thereafter providing additions to and/or furtherdetails in subsequent flowcharts) generally allows for a rapid and easyunderstanding of the various process implementations. In addition, thoseskilled in the art will further appreciate that the style ofpresentation used herein also lends itself well to modular and/orobject-oriented program design paradigms.

More particularly, in FIG. 9 and in the figures to follow thereafter,various operations may be depicted in a box-within-a-box manner. Suchdepictions may indicate that an operation in an internal box maycomprise an optional example embodiment of the operational stepillustrated in one or more external boxes. However, it should beunderstood that internal box operations may be viewed as independentoperations separate from any associated external boxes and may beperformed in any sequence with respect to all other illustratedoperations, or may be performed concurrently. Still further, theseoperations illustrated FIG. 9 as well as the other operations to bedescribed herein may be performed by at least one of a machine, anarticle of manufacture, or a composition of matter.

In any event, after a start operation, the operational flow 900 of FIG.9 may move to a user reaction detecting operation 902 for detecting oneor more user reactions of a user in response to a display to the user ofan augmented view of an actual scene from a real environment, theaugmented view that was displayed including one or more augmentations.For instance, and as illustration, the user reaction ascertaining module102* of FIG. 7A or 7B (e.g., the user reaction ascertaining module 102′of FIG. 7A or the user reaction ascertaining module 102″ of FIG. 7B)detecting or ascertaining one or more user reactions (e.g., ocular orfacial reactions) of a user 52 (see FIGS. 5A and 5B) in response to adisplay to the user 52 of an augmented view 56 (see FIG. 5A or 5B) of anactual scene (e.g., actual view 51 of FIG. 5A or 5B) from a realenvironment 50, the augmented view 56 that was displayed including oneor more augmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d,64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

As further illustrated in FIG. 9, operational flow 900 may also includea user reaction correlating operation 904 for correlating the detectedone or more user reactions with at least one or more aspects associatedwith the one or more augmentations that were included in the augmentedview that was presented. For instance, the user reaction associatingmodule 104* (e.g., the user reaction associating module 104′ of FIG. 7Aor the user reaction associating module 104″ of FIG. 7B) of the ARdevice 70* correlating the detected one or more user reactions of a user52 (e.g., eye movements and/or dwell time of the one or more eyes of theuser 52 with respect to the one or more augmentations included in thedisplayed augmented view 56, and/or other physiological characteristicsof the user 52 that may indicate, for example, mental state of the user52) with at least one or more aspects associated with the one or moreaugmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e,64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) that wereincluded in the augmented view 56 that was presented. In variousimplementations, the correlating of the detected one or more userreactions of the user 52 with the at least one or more aspectsassociated with the one or more augmentations may actually involve theassociating or linking of the detected one or more user reactions of theuser 52 with the at least one or more aspects associated with the one ormore augmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M). Notethat the various correlating operations to be described herein mayinvolve associating or linking of two variables (e.g., associating orlinking of one or more augmentations that were displayed to a user withone or more user reactions (e.g., eye focus or movements) that weredetected proximate to (e.g., sensed during and/or following) the displayof the one or more augmentations.

As will be described below, the user reaction detecting operation 902and the user reaction correlating operation 904 of FIG. 9 may beexecuted in a variety of different ways in various alternativeimplementations. FIGS. 10A, 10B, 10C, and 10D, for example, illustratesat least some of the alternative ways that the user reaction detectingoperation 902 of FIG. 9 may be executed in various alternativeimplementations. For example, FIGS. 10A, 10B, and 10C illustrate howvarious types of user reactions that may be detected (e.g., ascertained)through the user reaction detecting operation 902 of FIG. 9 in variousalternative implementations. In some cases, for example, the userreaction detecting operation 902 may include an operation 1002 fordetecting the one or more user reactions of the user including sensingone or more physiological characteristics of the user proximate to thedisplay of the augmented view as illustrated in FIG. 10A. For instance,the physiological characteristic determining module 802 (see FIG. 8A) ofthe AR device 70* (e.g., the AR device 70′ of FIG. 7A or the AR device70″ of FIG. 7B) detecting the one or more user reactions of the user 52including sensing (e.g., determining) one or more physiologicalcharacteristics (e.g., ocular characteristics. galvanic skin response,blood pressure, and/or pulse rate) of the user 52 proximate (e.g.,during and/or immediately after) to the display of the augmented view(see, for example, augmented view 60 b of FIG. 6B).

As further illustrated in FIG. 10A, operation 1002 may additionallyinclude one or more additional operations in various alternativeimplementations including in some cases an operation 1003 for sensingthe one or more physiological characteristics of the user includingsensing one or more eye characteristics of one or more eyes of the userduring and/or following the display of the augmented view. For instance,the physiological characteristic determining module 802 including theeye characteristic determining module 804 (see FIG. 8A) of the AR device70* of FIG. 7A or 7B sensing the one or more physiologicalcharacteristics of the user 52 when the eye characteristic determiningmodule 804 senses (e.g., visually or optically determining) one or moreeye characteristics of one or more eyes of the user 52 during and/orimmediately following the display of the augmented view (e.g., augmentedview 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D,6E, 6F, 6G, 6I, 6K, or 6M).

In various implementations, operation 1003 may further include anoperation 1004 for sensing the one or more eye characteristics of theone or more eyes of the user including sensing one or more eye movementsof the one or more eyes of the user during and/or following the displayof the augmented view. For instance, the eye characteristic determiningmodule 804 including the eye movement determining module 806 (see FIG.8A) of the AR device 70* of FIG. 7A or 7B sensing (e.g., determining)the one or more eye characteristics of the one or more eyes of the userwhen the eye movement determining module 806 senses one or more eyemovements of the one or more eyes of the user 52 during and/orimmediately following the display of the augmented view (see, forexample, augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M). For example, using one ormore cameras 870 and using an eye tracking application, movements of oneor more eyes of the user 52 may be tracked.

As further illustrated in FIG. 10A, operation 1004 may further includeone or more additional operations including in some cases, for example,an operation 1005 for sensing the one or more eye movements of the oneor more eyes of the user including tracking dwell path of the one ormore eyes of the user with respect to the augmented view during and/orfollowing the display of the augmented view. For instance, the eyemovement determining module 806 including the eye dwell path detectingmodule 808 (see FIG. 8A) of the AR device 70* of FIG. 7A or 7B sensing(e.g., detecting or determining) the one or more eye movements of theone or more eyes of the user 52 when the eye dwell path detecting module808 tracks dwell path 66 a (see FIG. 6C) of the one or more eyes of theuser 52 with respect to the augmented view 60 b during and/orimmediately following the display of the augmented view (see 60 b ofFIGS. 6B and 6C). Note that FIG. 6C illustrates a dwell path 66 a of auser 52 relative to the augmented view 60 b (e.g., with respect to theaugmented view 60 b).

In some cases, operation 1005 may further include an operation 1006 fortracking the dwell path of the one or more eyes of the user with respectto the augmented view including tracking dwell path of the one or moreeyes of the user as focus of the one or more eyes of the user moves awayfrom the one or more augmentations included in the augmented view. Forinstance, the eye dwell path detecting module 808 of the AR device 70*of FIG. 7A or 7B tracking the dwell path 66 a (see FIG. 6C) of the oneor more eyes of the user 52 with respect to the augmented view 60 b (seeFIG. 6C) including tracking dwell path 66 a of the one or more eyes ofthe user 52 as focus of the one or more eyes of the user 52 moves awayfrom the one or more augmentations 64 a (see the sundial augmentation 64a illustrated in FIGS. 6B and 6C) included in the augmented view 60 b.

In some cases, operation 1005 may additionally or alternative include orinvolve an operation 1007 for tracking the dwell path of the one or moreeyes of the user with respect to the augmented view including trackingdwell path of the one or more eyes of the user as focus of the one ormore eyes of the user moves between multiple augmentations included inthe augmented view as further depicted in FIG. 10A. For instance, theeye dwell path detecting module 808 of the AR device 70* of FIG. 7A or7B tracking the dwell path 66 b (see FIG. 6G) of the one or more eyes ofthe user 52 with respect to the augmented view 60 g (see FIG. 6G)including tracking dwell path 66 b of the one or more eyes of the user52 as focus of the one or more eyes of the user 52 moves betweenmultiple augmentations (e.g., augmentations 64 c and 64 e of FIG. 6G)included in the augmented view 60 g.

As illustrated in FIG. 10A, in some implementations operation 1003 forsensing the one or more physiological characteristics of the userincluding sensing one or more eye characteristics of one or more eyes ofthe user during and/or following the display of the augmented view mayadditionally or alternatively include or involve an operation 1008 forsensing the one or more eye characteristics of the one or more eyes ofthe user including sensing eye focus of the one or more eyes of the userwith respect to the augmented view during and/or following the displayof the augmented view. For instance, the eye characteristic determiningmodule 804 including the eye attention determining module 810 (see FIG.8A) of the AR device 70* of FIG. 7A or 7B sensing the one or more eyecharacteristics of the one or more eyes of the user 52 when the eyeattention determining module 810 senses eye focus 65 b (see FIG. 6E) ofthe one or more eyes of the user 52 with respect to the augmented view60 e (see FIG. 6E) during and/or immediately following the display ofthe augmented view 60 e.

As further illustrated in FIG. 10A, in various implementations operation1008 may involve one or more additional operations including in somecases an operation 1009 for sensing the eye focus of the one or moreeyes of the user including sensing eye focus of the one or more eyes ofthe user with respect to the one or more aspects of the one or moreaugmentations during and/or following the display of the augmented view.For instance, the eye attention determining module 810 of the AR device70* of FIG. 7A or 7B sensing the eye focus of the one or more eyes ofthe user 52 including sensing eye focus 65 c (see FIG. 6F) of the one ormore eyes of the user 52 with respect to the one or more aspects 67 a(e.g., the boarder or rim of the augmentation 64 c of FIG. 6F having aparticular color or lighting or flashing pattern) of the one or moreaugmentations 64 c during and/or immediately following the display ofthe augmented view 60 f. Note that in the example illustrated in FIG.6F, the one or more eyes of the user 52 instead of being focused (e.g.,eye focus 65 c) on the border of the augmentation 64 c may alternativelybe focused or dwell on items in the augmented view 60 f including, forexample, the information contained in the augmentation 64 c (whichindicates that there is a 90 percent change of precipitation), which mayultimately cause the user 52 to look upward towards the sky asillustrated in FIG. 6E.

In the same or alternative implementations, operation 1008 mayadditionally or alternatively include an operation 1010 for sensing theeye focus of the one or more eyes of the user including sensing eyefocus of the one or more eyes of the user with respect to the one ormore aspects of one or more non-augmentation scene elements included inthe augmented view during and/or following the display of the augmentedview. For instance, the eye attention determining module 810 of the ARdevice 70* of FIG. 7A or 7B sensing (e.g., detecting or determining) theeye focus of the one or more eyes of the user 52 including sensing eyefocus of the one or more eyes of the user 52 with respect to the one ormore aspects of one or more non-augmentation scene elements (e.g., theocean or sky illustrated in FIG. 6F) included in the augmented view 60 f(see FIG. 6F) during and/or immediately following the display of theaugmented view 60 f.

In the same or alternative implementations, operation 1008 mayadditionally or alternatively include an operation 1011 for sensing theeye focus of the one or more eyes of the user including sensing dwelltime of the one or more eyes of the user with respect to the one or moreaspects of the augmented view during and/or following the display of theaugmented view. For instance, the eye attention determining module 810including the eye attention time determining module 812 of the AR device70* of FIG. 7A or 7B sensing the eye focus of the one or more eyes ofthe user 52 including sensing dwell time of the one or more eyes of theuser 52 with respect to the one or more aspects (e.g., augmentation 64 fof FIG. 6I) of the augmented view 60 i (see FIG. 6I) during and/orimmediately following the display of the augmented view 60 i. Note thatwith respect to the example augmented view 60 i illustrated in FIG. 6Ithat may be displayed to the user 52, the user 52 may be bothered byhaving an augmentation 64 f placed at the center of his or her field ofview and therefore, may not want to dwell on the augmentation 64 f for along time, but instead, may look away from the augmentation 64 fquickly.

Turning now to FIG. 10B, in various implementations, operation 1003 forsensing the one or more physiological characteristics of the userincluding sensing one or more eye characteristics of one or more eyes ofthe user during and/or following the display of the augmented view mayactually involve or include an operation 1012 for sensing the one ormore eye characteristics of the one or more eyes of the user includingsensing one or more pupil characteristics of the one or more eyes of theuser during and/or following the display of the augmented view. Forinstance, the eye characteristic determining module 804 including thepupil characteristic detecting module 814 (see FIG. 8A) of the AR device70* of FIG. 7A or 7B sensing the one or more eye characteristics of theone or more eyes of the user 52 when the pupil characteristic detectingmodule 814 senses one or more pupil characteristics of the one or moreeyes of the user 52 during and/or immediately following the display ofthe augmented view (e.g., augmented view 60 b of FIG. 6B). That is, bysensing the size and shape of the pupils of a user 52, the user'sintensity of interest with respect to the augmented view that the user52 is viewing may ascertained.

As further illustrated in FIG. 10B, in various implementations operation1012 may further involve one or more additional operations including, insome cases an operation 1013 for sensing the one or more pupilcharacteristics of the one or more eyes of the user including sensingshape and/or size of one or more pupils of the user during and/orfollowing the display of the augmented view. For instance, the pupilcharacteristic detecting module 814 of the AR device 70* of FIG. 7A or7B sensing using, for example, one or more cameras 870 the one or morepupil characteristics of the one or more eyes of the user 52 includingsensing shape and/or size of one or more pupils of the user 52 duringand/or immediately following the display of the augmented view (e.g.,augmented view 60 d of FIG. 6D).

In the same or alternative implementations, the operation 1012 mayadditionally or alternatively include an operation 1014 for sensing theone or more pupil characteristics of the one or more eyes of the userincluding sensing the one or more pupil characteristics in combinationwith sensing eye focus of the one or more eyes of the user with respectto the augmented view during and/or following the display of theaugmented view. For instance, the pupil characteristic detecting module814 and the eye attention determining module 810 of the AR device 70* ofFIG. 7A or 7B sensing the one or more pupil characteristics of the oneor more eyes of the user 52 including sensing, by the pupilcharacteristic detecting module 814, of the one or more pupilcharacteristics of the user 52 in combination with sensing, by the eyeattention determining module 810, of the eye focus (see eye focus 65 cof FIG. 6F) of the one or more eyes of the user 52 with respect to theaugmented view 60 f during and/or immediately following the display ofthe augmented view (e.g., augmented view 60 f of FIG. 6F). That is, bysensing the size and shape of the pupils of a user 52 along with sensingwhat visual item is the user visually focused on, the user's intensityof interest with respect to the visual item (e.g. an augmentation or anon-augmentation visual item) that the user is visually focused on mayat least be inferred or estimated.

In some implementations, operation 1014 may further include or involvean operation 1015 for sensing the one or more pupil characteristics incombination with sensing the eye focus of the one or more eyes of theuser including sensing the one or more pupil characteristics incombination with sensing eye focus of the one or more eyes of the userwith respect to the one or more aspects of the one or more augmentationsduring and/or following the display of the augmented view. For instance,the pupil characteristic detecting module 814 and the eye attentiondetermining module 810 of the AR device 70* of FIG. 7A or 7B sensing theone or more pupil characteristics in combination with sensing the eyefocus of the one or more eyes of the user 52 including sensing, by thepupil characteristic detecting module 814, of the one or more pupilcharacteristics in combination with sensing, by the eye attentiondetermining module 810, of eye focus of the one or more eyes of the user52 with respect to the one or more aspects of the one or moreaugmentations (e.g., augmentation aspect 67 a of FIG. 6F, which is theboarder to augmentation 64 cc of FIG. 6F) during and/or immediatelyfollowing the display of the augmented view 60 f.

In the same or different implementations, the operation 1014 mayadditionally or alternatively include an operation 1016 for sensing theone or more pupil characteristics in combination with sensing the eyefocus of the one or more eyes of the user including sensing the one ormore pupil characteristics in combination with sensing eye focus of theone or more eyes of the user with respect to the one or more aspects ofone or more non-augmentation scene elements included in the augmentedview during and/or following the display of the augmented view. Forinstance, the pupil characteristic detecting module 814 and the eyeattention determining module 810 of the AR device 70* of FIG. 7A or 7Bsensing the one or more pupil characteristics in combination withsensing the eye focus of the one or more eyes of the user 52 includingsensing, by the pupil characteristic detecting module 814, of the one ormore pupil characteristics in combination with sensing, by the eyeattention determining module 810, of eye focus (e.g., eye focus 65 b ofFIG. 6E) of the one or more eyes of the user 52 with respect to one theor more aspects of one or more non-augmentation scene elements (e.g.,the sky illustrated in FIG. 6E) included in the augmented view 60 e (seeFIG. 6E) during and/or immediately following the display of theaugmented view 60 e.

In some cases, operation 1012 for sensing the one or more eyecharacteristics of the one or more eyes of the user including sensingone or more pupil characteristics of the one or more eyes of the userduring and/or following the display of the augmented view may actuallyinvolve or include one or more additional operations including anoperation 1017 for sensing the one or more pupil characteristics of theone or more eyes of the user including sensing the one or more pupilcharacteristics in combination with tracking dwell path of the one ormore eyes of the user with respect to the augmented view during and/orfollowing the display of the augmented view. For instance, the pupilcharacteristic detecting module 814 and the eye dwell path detectingmodule 808 of the AR device 70* of FIG. 7A or 7B sensing the one or morepupil characteristics of the one or more eyes of the user 52 includingsensing, by the pupil characteristic detecting module 814, of the one ormore pupil characteristics of the user 52 in combination with tracking,by the eye dwell path detecting module 808, of the dwell path 66 b (seeFIG. 6E) of the one or more eyes of the user 52 with respect to theaugmented view 60 e (see FIG. 6E) during and/or immediately followingthe display of the augmented view 60 e.

Turning now to FIG. 10C, in various implementations, operation 1002 fordetecting the one or more user reactions of the user including sensingone or more physiological characteristics of the user proximate to thedisplay of the augmented view may additionally include or involve anoperation 1018 for sensing the one or more physiological characteristicsof the user including sensing one or more cardiopulmonarycharacteristic, skin characteristic and/or brain characteristic of theuser during and/or following the display of the augmented view. Forinstance, the physiological characteristic determining module 802 of theAR device 70* of FIG. 7A or 7B sensing the one or more physiologicalcharacteristics of the user including sensing one or morecardiopulmonary characteristic, skin characteristic and/or braincharacteristic of the user 52 during and/or immediately following thedisplay of the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).In some implementations, such physiological characteristics may becollected and processed in order to infer the mental and/or physicalstate of the user 52. Based on the determined mental and/or physicalstate of the user 52, the most appropriate augmentation, as well as itsformat, may be selected for presentation by for example, the AR device70*.

In the same or different implementations, operation 1002 mayadditionally or alternatively include an operation 1019 for sensing theone or more physiological characteristics of the user proximate to thedisplay of the augmented view including sensing the one or morephysiological characteristics of the user at least just prior to thedisplay of the augmented view. For instance, the physiologicalcharacteristic determining module 802 of the AR device 70* of FIG. 7A or7B sensing the one or more physiological characteristics of the userproximate to the display of the augmented view including sensing the oneor more physiological characteristics (e.g., galvanic skin response,heart rate, and so forth) of the user 52 at least just prior to thedisplay of the augmented view 60 b (see FIG. 6B). Such an operation maybe executed in some cases in order to, for example, provide baselinephysiological characteristics. That is, in order to determine what userphysiological characteristic (e.g., changes in user physiologicalcharacteristics) can be attributed to the presentation or display of anaugmented view (e.g., augmented view 60 i of FIG. 6I), the baselinephysiological characteristics of the user 52 when the user 52 is not“stimulated” by the augmented view may be determined.

In some cases, operation 1019 may further include an operation 1020 forsensing the one or more physiological characteristics of the user atleast just prior to the display of the augmented view including sensingone or more eye characteristics, one or more cardiopulmonarycharacteristics, one or more skin characteristics, and/or one or morebrain characteristics of the user at least just prior to the display ofthe augmented view. For instance, the physiological characteristicdetermining module 802 of the AR device 70* of FIG. 7A or 7B sensing theone or more physiological characteristics of the user at least justprior to the display of the augmented view including sensing one or moreeye characteristics, one or more cardiopulmonary characteristics, one ormore skin characteristics, and/or one or more brain characteristics(e.g., as sensed by a functional near infrared (fNIR) device orfunctional magnetic resonance imaging (fMRI) device) of the user 52 atleast just prior to the display of the augmented view (e.g., augmentedview 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D,6E, 6F, 6G, 6I, 6K, or 6M).

In various implementations, the user reaction detecting operation 902 ofFIG. 9 may include an operation 1021 for detecting the one or more userreactions of the user including detecting at least one of facialexpression, head position, or head tilt of the user proximate to thedisplay of the augmented view as further illustrated in FIG. 10C. Forinstance, the user reaction ascertaining module 102* of the AR device70* of FIG. 7A or 7B detecting the one or more user reactions of theuser 52 including detecting at least one of facial expression, headposition, or head tilt (e.g., as detected using one or more cameras 870,tilt sensors, and/or other sensors) of the user proximate to the displayof the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60g, 60 i, or 60 k of FIG. 6B, 6D, 6E, 6F, 6G, 6I, or 6K).

As further illustrated in FIG. 10C, operation 1021 in someimplementations may further include or involve an operation 1022 fordetecting at least one of the facial expression, the head position, orthe head tilt of the user during and/or following the display of theaugmented view. For instance, the user reaction ascertaining module 102*of the AR device 70* of FIG. 7A or 7B detecting at least one of thefacial expression, the head position, or the head tilt of the user 52during and/or immediately following the display of the augmented view(e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 i, or 60 k of FIG. 6B,6D, 6E, 6F, 6I, or 6K). In some implementations, such an operation maybe implemented in order to provide a baseline characteristic of the user52.

In the same or different implementations, the user reaction detectingoperation 902 may additionally or alternatively include an operation1023 for detecting the one or more user reactions of the user inresponse to the display of the augmented view including registering oneor more aspects of the augmented view. For instance, the user reactionascertaining module 102* including the augmented view aspect recordingmodule 816 (see FIG. 8A) of the AR device 70* of FIG. 7A or 7B detectingthe one or more user reactions (e.g., eye focus or movements) of theuser 52 in response by the user 52 to the display of the augmented view(e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, or 60 k ofFIG. 6B, 6D, 6E, 6F, 6G, 6I, or 6K) when the augmented view aspectrecording module 816 registers (e.g., records or chronicles) one or moreaspects of the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60f, 60 g, 60 i, or 60 k of FIG. 6B, 6D, 6E, 6F, 6G, 6I, or 6K).

As further depicted in FIG. 10C, in some implementations operation 1023may further include or involve an operation 1024 for registering the oneor more aspects of the presented augmented view including registeringone or more forms of the one or more augmentations included in theaugmented view. For instance, the augmented view aspect recording module816 including the augmentation form recording module 818 (see FIG. 8A)of the AR device 70* of FIG. 7A or 7B registering the one or moreaspects of the presented augmented view when the augmentation formrecording module 818 registers one or more forms (e.g., color, shading,placement in the augmented view, etc.) of the one or more augmentations(e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) included in theaugmented view (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i,60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In some cases, operation 1024 may further include an operation 1025 forregistering the one or more forms of the one or more augmentationsincluding registering one or more shapes and/or dimensions of the one ormore augmentations. For instance, the augmentation form recording module818 including the augmentation shape/dimension recording module 820 (seeFIG. 8A) of the AR device 70* of FIG. 7A or 7B registering the one ormore forms of the one or more augmentations when the augmentationshape/dimension recording module 820 registers one or more shapes and/ordimensions of the one or more augmentations (e.g., augmentation 64 a, 64b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F,6G, 6I, 6K, or 6M).

In the same or different implementations, operation 1024 mayadditionally or alternatively include an operation 1026 for registeringthe one or more forms of the one or more augmentations includingregistering one or more locations of the one or more augmentations inthe augmented view. For instance, the augmentation form recording module818 including the augmentation location recording module 222 (see FIG.8A) of the AR device 70* of FIG. 7A or 7B registering the one or moreforms of the one or more augmentations when the augmentation locationrecording module 222 registers one or more locations of the one or moreaugmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e,64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) in theaugmented view (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i,60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

Referring now to FIG. 10D, in various implementations the operation 1023for detecting the one or more user reactions of the user in response tothe display of the augmented view including registering one or moreaspects of the augmented view may in some cases include an operation1027 for registering the one or more aspects of the presented augmentedview including registering one or more aspects of one or morenon-augmentation scene elements included in the augmented view. Forinstance, the augmented view aspect recording module 816 including thenon-augmentation scene element aspect recording module 824 (see FIG. 8A)of the AR device 70*of FIG. 7A or 7B registering the one or more aspectsof the presented augmented view when the non-augmentation scene elementaspect recording module 824 registers one or more aspects (e.g., color,size, placement in the augmented view, and so forth) of one or morenon-augmentation scene elements (e.g., a sign, a store or buildingfront, a street scene, and so forth) included in the augmented view(e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 mof FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M). In various implementations,the registration of non-augmentation scene elements may be needed inorder to take into account the context in which the user 52 reacts to anaugmentation. That is, a user 52 may react to an augmentation (e.g.,augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h ofFIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) differently depending on thecontext in which the augmentation is presented. The “context” referredto here is in reference to scene elements such as non-augmentation sceneelements that may be presented in the corresponding augmented view(e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 mof FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

As further illustrated in FIG. 10D, in various implementations operation1027 may further include one or more additional operations including, insome cases, an operation 1028 for registering the one or more aspects ofthe one or more non-augmentation scene elements including registeringone or more locations of the one or more non-augmentation scene elementsin the augmented view. For instance, the non-augmentation scene elementaspect recording module 824 including the non-augmentation scene elementlocation recording module 826 (see FIG. 8A) of the AR device 70*of FIG.7A or 7B registering the one or more aspects of the one or morenon-augmentation scene elements when the non-augmentation scene elementlocation recording module 826 registers one or more locations of the oneor more non-augmentation scene elements in the augmented view (e.g.,augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG.6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In some cases, operation 1028 may further include an operation 1029 forregistering the one or more aspects of the one or more non-augmentationscene elements including registering the one or more locations of theone or more non-augmentation scene elements relative to one or morelocations of the one or more augmentations in the augmented view. Forinstance, the non-augmentation scene element aspect recording module 824including the non-augmentation scene element location recording module826 of the AR device 70*of FIG. 7A or 7B registering the one or moreaspects of the one or more non-augmentation scene elements when thenon-augmentation scene element location recording module 826 registers(e.g., records) the one or more locations of the one or morenon-augmentation scene elements relative to one or more locations of theone or more augmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc,64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or6M) in the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60 f,60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In some cases, operation 1023 may additionally or alternatively includean operation 1030 for registering the one or more aspects of thepresented augmented view including registering one or morenon-augmentation scene elements that have been obfuscated in theaugmented view by the one or more augmentations. For instance, theaugmented view aspect recording module 816 including the obfuscatednon-augmentation scene element recording module 828 (see FIG. 8A) of theAR device 70*of FIG. 7A or 7B registering the one or more aspects of thepresented augmented view when the obfuscated non-augmentation sceneelement recording module 828 registers one or more non-augmentationscene elements that have been obfuscated (e.g., hidden) in the augmentedview (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) by the one or moreaugmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e,64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M). Such anoperation may be particularly beneficial when, for example, obfuscationof a non-augmentation scene element causes the user 52 to react in aparticular measurable way (e.g., eye movement or eye focus).

Referring back to the user reaction correlating operation 904 of FIG. 9,the user reaction correlating operation 904 similar to the user reactiondetecting operation 902 of FIG. 9 may be executed in a number ofdifferent ways in various alternative embodiments as illustrated inFIGS. 11A, 11B, 11C, 11D, and 11E. In some implementations, for example,the user reaction correlating operation 904 may include an operation1131 for correlating the detected one or more user characteristics withthe at least one or more aspects associated with the one or moreaugmentations by defining one or more relationships between the detectedone or more user reactions and the at least one or more aspectsassociated with the one or more augmentations. For instance, the userreaction associating module 104* including the relationship identifyingmodule 830 (see FIG. 8B) of the AR device 70* of FIG. 7A or 7Bcorrelating (e.g., associating, linking, connecting, and so forth) thedetected one or more user characteristics with the at least one or moreaspects associated with the one or more augmentations when therelationship identifying module 830 defines (e.g., identifies ordetermines) one or more relationships between the detected one or moreuser reactions (e.g., eye movements or eye focus) and the at least oneor more aspects (e.g., color, shading, placement in the augmented view,and so forth) associated with the one or more augmentations (e.g.,augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h ofFIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In some cases, operation 1131 may further include an operation 1132 forcorrelating the detected one or more user characteristics with the atleast one or more aspects associated with the one or more augmentationsby registering the defined one or more relationships between thedetected one or more user characteristics and the at least one or moreaspects associated with the one or more augmentations. For instance, theuser reaction associating module 104* including the relationship loggingmodule 832 (see FIG. 8B) of the AR device 70* of FIG. 7A or 7Bcorrelating the detected one or more user characteristics (e.g., eyemovements or focus) with the at least one or more aspects (e.g., color,bordering, brightness, and so forth) associated with the one or moreaugmentations when the relationship logging module 832 registers thedefined one or more relationships between the detected one or more usercharacteristics (e.g., eye movements, eye focus, dwell path, dwell time,and so forth) and the at least one or more aspects (e.g., size andcolor) associated with the one or more augmentations (e.g., augmentation64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D,6E, 6F, 6G, 6I, 6K, or 6M).

In the same or different implementations, the user reaction correlatingoperation 904, as further illustrated in FIG. 11A, may additionally oralternatively include an operation 1133 for correlating the detected oneor more user reactions with the at least one or more aspects associatedwith the one or more augmentations by correlating one or morephysiological characteristics of the user that were sensed during and/orfollowing the display of the augmented view with the one or more aspectsof the one or more augmentations. For instance, the user reactionassociating module 104* including the physiological characteristicassociating module 834 (see FIG. 8B) of the AR device 70* of FIG. 7A or7B correlating the detected one or more user reactions with the at leastone or more aspects associated with the one or more augmentations whenthe physiological characteristic associating module 834 correlates oneor more physiological characteristics (e.g., eye characteristics, facialcharacteristics, skin characteristics, and so forth) of the user 52 thatwere sensed during and/or immediately following the display of theaugmented view (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i,60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) with the one ormore aspects of the one or more augmentations (e.g., augmentation 64 a,64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E,6F, 6G, 6I, 6K, or 6M).

As further illustrated in FIG. 11A, in various implementations operation1133 may further include one or more additional operations. For example,in some cases, operation 1133 may include an operation 1134 forcorrelating the one or more physiological characteristics of the userwith the one or more aspects of the one or more augmentations bycorrelating one or more eye characteristics of one or more eyes of theuser that were sensed during and/or following the display of theaugmented view with the one or more aspects of the one or moreaugmentations. For instance, the physiological characteristicassociating module 834 including the eye characteristic associatingmodule 836 (see FIG. 8B) of the AR device 70* of FIG. 7A or 7Bcorrelating the one or more physiological characteristics of the userwith the one or more aspects of the one or more augmentations when theeye characteristic associating module 836 correlates (e.g., associatesor links) one or more eye characteristics of one or more eyes of theuser 52 that were sensed during and/or immediately following the displayof the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) withthe one or more aspects of the one or more augmentations (e.g.,augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h ofFIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In some implementations, operation 1134 may further include or involvean operation 1135 for correlating the one or more eye characteristics ofthe one or more eyes of the user with the one or more aspects of the oneor more augmentations by correlating one or more eye movements of theone or more eyes of the user that were sensed during and/or followingthe display of the augmented view with the one or more aspects of theone or more augmentations. For instance, the eye characteristicassociating module 836 including the eye movement associating module 838(see FIG. 8B) of the AR device 70* of FIG. 7A or 7B correlating the oneor more eye characteristics of the one or more eyes of the user 52 withthe one or more aspects of the one or more augmentations when the eyemovement associating module 838 correlates one or more eye movements ofthe one or more eyes of the user 52 that were sensed during and/orimmediately following the display of the augmented view (e.g., augmentedview 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D,6E, 6F, 6G, 6I, 6K, or 6M) with the one or more aspects of the one ormore augmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

As further depicted in FIG. 11A, operation 1135 may in some casesinclude an operation 1136 for correlating the sensed one or more eyemovements of the one or more eyes of the user with the one or moreaspects of the one or more augmentations by correlating dwell path ofthe one or more eyes of the user that was tracked with respect to theaugmented view during and/or following the display of the augmented viewwith the one or more aspects of the one or more augmentations. Forinstance, the eye movement associating module 838 including the dwellpath associating module 840 (see FIG. 8B) of the AR device 70* of FIG.7A or 7B correlating the sensed one or more eye movements of the one ormore eyes of the user sensed with the one or more aspects of the one ormore augmentations when the dwell path associating module 840 correlates(e.g., associates) dwell path of the one or more eyes of the user 52that was tracked with respect to the augmented view (e.g., augmentedview 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D,6E, 6F, 6G, 6I, 6K, or 6M) during and/or immediately following thedisplay of the augmented view with the one or more aspects of the one ormore augmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In some cases, operation 1136 may include one or more additionaloperations including in some cases an operation 1137 for correlating thedwell path of the one or more eyes of the user with the one or moreaspects of the one or more augmentations by correlating dwell path ofthe one or more eyes of the user that was tracked as focus of the one ormore eyes of the user moved away from the one or more augmentationsincluded in the augmented view with the one or more aspects of the oneor more augmentations. For instance, the dwell path associating module840 of the AR device 70* of FIG. 7A or 7B correlating (e.g., linking orassociating) the dwell path of the one or more eyes of the user 52 withthe one or more aspects of the one or more augmentations by correlatingdwell path (e.g., dwell path 66 b of FIG. 6E) of the one or more eyes ofthe user 52 that was tracked as focus of the one or more eyes of theuser 52 moved away from the one or more augmentations (e.g.,augmentation 64 c of FIG. 6E) included in the augmented view 60 e (seeFIG. 6E) with the one or more aspects of the one or more augmentations64 c.

In the same or alternative implementations, operation 1136 mayadditionally or alternatively include an operation 1138 for correlatingthe dwell path of the one or more eyes of the user with the one or moreaspects of the one or more augmentations by correlating dwell path ofthe one or more eyes of the user that was tracked as focus of the one ormore eyes of the user moved between multiple augmentations included inthe augmented view with the one or more aspects of the one or moreaugmentations. For instance, the dwell path associating module 840 ofthe AR device 70* of FIG. 7A or 7B correlating the dwell path of the oneor more eyes of the user 52 with the one or more aspects of the one ormore augmentations by correlating dwell path 66 b (see FIG. 6G) of theone or more eyes of the user 52 that was tracked as focus of the one ormore eyes of the user 52 moved between multiple augmentations (e.g.,augmentations 64 c and 64 e of FIG. 6G) included in the augmented view60 g (see FIG. 6G) with the one or more aspects of the one or moreaugmentations (e.g., augmentation 64 c and/or augmentation 64 e).

Referring now to FIG. 11B, in various implementations the operation 1134for correlating the one or more physiological characteristics of theuser with the one or more aspects of the one or more augmentations bycorrelating one or more eye characteristics of one or more eyes of theuser that were sensed during and/or following the display of theaugmented view with the one or more aspects of the one or moreaugmentations may additionally include an operation 1139 for correlatingthe one or more eye characteristics of the one or more eyes of the userwith the one or more aspects of the one or more augmentations bycorrelating eye focus of the one or more eyes of the user with respectto the augmented view that were sensed during and/or following thedisplay of the augmented view with the one or more aspects of the one ormore augmentations. For instance, the eye characteristic associatingmodule 836 including the eye focus associating module 842 (see FIG. 8B)of the AR device 70* of FIG. 7A or 7B correlating the one or more eyecharacteristics of the one or more eyes of the user 52 with the one ormore aspects of the one or more augmentations when the eye focusassociating module 842 correlates or associates eye focus 65 a (see FIG.6F) of the one or more eyes of the user 52 with respect to the augmentedview (e.g., augmented view 60 f of FIG. 6F) that were sensed duringand/or immediately following (e.g., detected proximate to) the displayof the augmented view 60 f with the one or more aspects of the one ormore augmentations (e.g., augmentation 64 c and/or augmentation 64 d).In various implementations, the eye focus 65 a of the one or more eyesof the user 52 may have been detecting proximate to the display of theaugmented view 60 f using, for example, one or more cameras 870.

As further illustrated in FIG. 11B, in various implementations,operation 1139 may include one or more additional operations including,in some cases, an operation 1140 for correlating the eye focus of theone or more eyes of the user with the one or more aspects of the one ormore augmentations by correlating eye focus of the one or more eyes ofthe user with the one or more aspects of the one or more augmentations,the eye focus of the one or more eyes of the user to be correlated beingwith respect to the one or more aspects of the one or more augmentationsthat were sensed during and/or following the display of the augmentedview. For instance, the eye focus associating module 842 of the ARdevice 70* of FIG. 7A or 7B correlating the eye focus of the one or moreeyes of the user 52 with the one or more aspects of the one or moreaugmentations by correlating or associating eye focus of the one or moreeyes of the user 52 with the one or more aspects of the one or moreaugmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e,64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M), the eyefocus of the one or more eyes of the user 52 to be correlated being withrespect to the one or more aspects of the one or more augmentations(e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) that were sensed duringand/or immediately following (e.g., detected proximate to) the displayof the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In the same or alternative implementations, operation 1139 mayadditionally or alternatively include an operation 1141 for correlatingthe eye focus of the one or more eyes of the user with the one or moreaspects of the one or more augmentations by correlating the eye focus ofthe one or more eyes of the user with the one or more aspects of the oneor more augmentations, the eye focus of the one or more eyes of the userbeing with respect to one or more aspects of one or morenon-augmentation scene elements included in the augmented view that weresensed during and/or following the display of the augmented view. Forinstance, the eye focus associating module 842 of the AR device 70* ofFIG. 7A or 7B correlating the eye focus of the one or more eyes of theuser 52 with the one or more aspects of the one or more augmentations bycorrelating (e.g., associating) the eye focus of the one or more eyes ofthe user 52 with the one or more aspects of the one or moreaugmentations, the eye focus of the one or more eyes of the user 52being with respect to one or more aspects of one or morenon-augmentation scene elements (e.g., the beach illustrated in theexample augmented view 60 c of FIG. 6C) included in the augmented view60 c that were sensed during and/or immediately following (e.g.,detected proximate to) the display of the augmented view 60 c.

In the same or alternative implementations, operation 1139 mayadditionally or alternatively include an operation 1142 for correlatingthe eye focus of the one or more eyes of the user with the one or moreaspects of the one or more augmentations by correlating dwell time ofthe one or more eyes of the user with the one or more aspects of the oneor more augmentations, the eye focus of the one or more eyes of the userbeing with respect to one or more aspects of the augmented view thatwere sensed during and/or following the display of the augmented view.For instance, the eye focus associating module 842 including the dwelltime associating module 844 (see FIG. 8B) of the AR device 70* of FIG.7A or 7B correlating the eye focus of the one or more eyes of the user52 with the one or more aspects of the one or more augmentations bycorrelating or associating dwell time (e.g., the amount of time a user52 is visually focused on or stares at a visual item) of the one or moreeyes of the user 52 with the one or more aspects of the one or moreaugmentations, the eye focus of the one or more eyes of the user 52being with respect to one or more aspects of the augmented view (e.g.,augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG.6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) that were sensed during and/orimmediately following (e.g., that were detected proximate to) thedisplay of the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

Turning now to FIG. 11C, in various implementations, operation 1134 forcorrelating the one or more physiological characteristics of the userwith the one or more aspects of the one or more augmentations bycorrelating one or more eye characteristics of one or more eyes of theuser that were sensed during and/or following the display of theaugmented view with the one or more aspects of the one or moreaugmentations may additionally or alternatively include an operation1143 for correlating the one or more eye characteristics of the one ormore eyes of the user with the one or more aspects of the one or moreaugmentations by correlating one or more pupil characteristics of theone or more eyes of the user that were sensed during and/or followingthe display of the augmented view with the one or more aspects of theone or more augmentations. For instance, the eye characteristicassociating module 836 including the pupil characteristic associatingmodule 846 (see FIG. 8B) of the AR device 70* of FIG. 7A or 7Bcorrelating the one or more eye characteristics of the one or more eyesof the user 52 with the one or more aspects of the one or moreaugmentations when the pupil characteristic associating module 846correlates one or more pupil characteristics of the one or more eyes ofthe user 52 that were sensed during and/or immediately following (e.g.,that were detected proximate to) the display of the augmented view(e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 mof FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) with the one or more aspectsof the one or more augmentations (e.g., augmentation 64 a, 64 b, 64 c,64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I,6K, or 6M).

As further depicted in FIG. 11C, operation 1143 may further include oneor more additional operations in various alternative implementations.For example, in some implementations, operation 1143 may include anoperation 1144 for correlating the one or more pupil characteristics ofthe one or more eyes of the user with the one or more aspects of the oneor more augmentations by correlating shape and/or size of one or morepupils of the user that were sensed during and/or following the displayof the augmented view with the one or more aspects of the one or moreaugmentations. For instance, the pupil characteristic associating module846 including the pupil shape/size associating module 848 (see FIG. 8B)of the AR device 70* of FIG. 7A or 7B correlating the one or more pupilcharacteristics of the one or more eyes of the user 52 with the one ormore aspects of the one or more augmentations when the pupil shape/sizeassociating module 848 correlates (e.g., associates) the shape and/orsize of one or more pupils of the user 52 that were sensed during and/orimmediately following (e.g., that were detected proximate to) thedisplay of the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M)with the one or more aspects of the one or more augmentations (e.g.,augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h ofFIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In the same or alternative implementations, operation 1143 mayadditionally or alternatively include an operation 1145 for correlatingthe one or more pupil characteristics of the one or more eyes of theuser with the one or more aspects of the one or more augmentations bycorrelating the one or more pupil characteristics in combination witheye focus of the one or more eyes of the user with the one or moreaspects of the one or more augmentations, the one or more pupilcharacteristics and the eye focus of the one or more eyes of the userbeing with respect to the augmented view that were sensed during and/orfollowing the display of the augmented view. For instance, the AR device70* of FIG. 7A or 7B correlating the one or more pupil characteristicsof the one or more eyes of the user 52 with the one or more aspects ofthe one or more augmentations when the pupil characteristic associatingmodule 846 and the eye focus associating module 842 of the AR device 70*correlates the one or more pupil characteristics in combination with eyefocus of the one or more eyes of the user 52 with the one or moreaspects of the one or more augmentations (e.g., augmentation 64 a, 64 b,64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G,6I, 6K, or 6M), the one or more pupil characteristics and the eye focusof the one or more eyes of the user 52 being with respect to theaugmented view (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i,60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) that weresensed during and/or immediately following the display of the augmentedview (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In some cases, operation 1145 may actually include or involve anoperation 1146 for correlating the one or more pupil characteristics incombination with the eye focus of the one or more eyes of the user withthe one or more aspects of the one or more augmentations by correlatingthe one or more pupil characteristics in combination with the eye focusof the one or more eyes of the user with the one or more aspects of theone or more augmentations, the one or more pupil characteristics and theeye focus of the one or more eyes of the user being with respect to theone or more aspects of the one or more augmentations that were sensedduring and/or following the display of the augmented view. For instance,the AR device 70* of FIG. 7A or 7B correlating the one or more pupilcharacteristics in combination with the eye focus of the one or moreeyes of the user with the one or more aspects of the one or moreaugmentations when the pupil characteristic associating module 846 andthe eye focus associating module 842 of the AR device 70* correlates theone or more pupil characteristics in combination with the eye focus ofthe one or more eyes of the user 52 with the one or more aspects of theone or more augmentations, the one or more pupil characteristics and theeye focus of the one or more eyes of the user 52 being with respect tothe one or more aspects of the one or more augmentations (e.g.,augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h ofFIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) that were sensed during and/orimmediately following (e.g., that were detected proximate to) thedisplay of the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).That is, in various embodiments, the interest of a user 52 with respectto one or more aspects of one or more augmentations (e.g., augmentation64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D,6E, 6F, 6G, 6I, 6K, or 6M) may be defined based on the eye focus and thepupil characteristics of the user 52 with respect to the one or moreaspects of the one or more augmentations (e.g., whether the user 52 islooking at the one or more aspects of the one or more augmentations andwhether the pupils of the user 52 are or are not dilated when the user52 is looking at the one or more aspects of the one or moreaugmentations).

In the same or alternative implementations, operation 1145 mayadditionally or alternatively include or involve an operation 1147 forcorrelating the one or more pupil characteristics in combination withthe eye focus of the one or more eyes of the user with the one or moreaspects of the one or more augmentations by correlating the one or morepupil characteristics in combination with the eye focus of the one ormore eyes of the user with the one or more aspects of the one or moreaugmentations, the one or more pupil characteristics and the eye focusof the one or more eyes of the user being with respect to one or moreaspects of one or more non-augmentation scene elements included in theaugmented view that was sensed during and/or following the display ofthe augmented view. For instance, the AR device 70* of FIG. 7A or 7Bcorrelating the one or more pupil characteristics in combination withthe eye focus of the one or more eyes of the user 52 with the one ormore aspects of the one or more augmentations when the pupilcharacteristic associating module 846 and the eye focus associatingmodule 842 of the AR device 70* correlates the one or more pupilcharacteristics in combination with eye focus of the one or more eyes ofthe user with the one or more aspects of the one or more augmentations(e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M), the one or more pupilcharacteristics and the eye focus of the one or more eyes of the user 52being with respect to one or more aspects of one or morenon-augmentation scene elements (e.g., store front, a sign, beach, andso forth) included in the augmented view (e.g., augmented view 60 b, 60d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I,6K, or 6M) that was sensed during and/or immediately following (e.g.,that were detected proximate to) the display of the augmented view.

Referring now to FIG. 11D, in some implementations, operation 1143 forcorrelating the one or more eye characteristics of the one or more eyesof the user with the one or more aspects of the one or moreaugmentations by correlating one or more pupil characteristics of theone or more eyes of the user that were sensed during and/or followingthe display of the augmented view with the one or more aspects of theone or more augmentations may actually include or involve an operation1148 for correlating the one or more pupil characteristics of the one ormore eyes of the user with the one or more aspects of the one or moreaugmentations by correlating the one or more pupil characteristics incombination with dwell path of the one or more eyes of the user with theone or more aspects of the one or more augmentations, the one or morepupil characteristics and the dwell path of the one or more eyes of theuser being with respect to the augmented view that was tracked duringand/or following the display of the augmented view. For instance, the ARdevice 70* of FIG. 7A or 7B correlating the one or more pupilcharacteristics of the one or more eyes of the user with the one or moreaspects of the one or more augmentations when the pupil characteristicassociating module 846 and the dwell path associating module 840 of theAR device 70* correlates the one or more pupil characteristics incombination with dwell path (e.g., dwell path 66 a of FIG. 6C) of theone or more eyes of the user 52 with the one or more aspects of the oneor more augmentations (e.g., augmentation 64 a of FIG. 6C), the one ormore pupil characteristics and the dwell path 66 a (see FIG. 6C) of theone or more eyes of the user 52 being with respect to the augmented view60 c (see FIG. 6C) that was tracked during and/or immediately following(e.g., that were detected or tracked proximate to) the display of theaugmented view 66 c.

As further illustrated in FIG. 11D, the operation 1133 for correlatingthe detected one or more user reactions with the at least one or moreaspects associated with the one or more augmentations by correlating oneor more physiological characteristics of the user that were sensedduring and/or following the display of the augmented view with the oneor more aspects of the one or more augmentations may additionally oralternatively include or involve an operation 1149 for correlating theone or more physiological characteristics of the user with the one ormore aspects of the one or more augmentations by correlating one or morecardiopulmonary characteristic, skin characteristic and/or braincharacteristic of the user that were sensed during and/or following thedisplay of the augmented view with the one or more aspects of the one ormore augmentations. For instance, the physiological characteristicassociating module 834 (see FIG. 8B) of the AR device 70* of FIG. 7A or7B correlating the one or more physiological characteristics of the user52 with the one or more aspects of the one or more augmentations bycorrelating (e.g., linking or associating) one or more cardiopulmonarycharacteristic, skin characteristic and/or brain characteristic of theuser 52 that were sensed during and/or immediately following (e.g., thatwere detected proximate to) the display of the augmented view (e.g.,augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG.6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) with the one or more aspects of theone or more augmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc,64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or6M).

In various implementations, the user reaction correlating operation 904for correlating the detected one or more user reactions with at leastone or more aspects associated with the one or more augmentations thatwere included in the augmented view that was presented, as furtherillustrated in FIG. 11D, may additionally or alternatively include orinvolve an operation 1150 for correlating the detected one or more userreactions with at least the one or more aspects associated with the oneor more augmentations by correlating at least one of a facialexpression, a head position, or a head tilt of the user that wasdetected during and/or following the display of the augmented view withthe one or more augmentations that were included in the augmented viewthat was displayed. For instance, the user reaction associating module104* of the AR device 70* of FIG. 7A or 7B correlating (e.g.,associating or linking) the detected one or more user reactions with atleast the one or more aspects associated with the one or moreaugmentations by correlating at least one of a facial expression, a headposition, or a head tilt of the user 52 that was detected during and/orimmediately following (e.g., that were detected proximate to) thedisplay of the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M)with the one or more augmentations (e.g., augmentation 64 a, 64 b, 64 c,64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I,6K, or 6M) that were included in the augmented view (e.g., augmentedview 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D,6E, 6F, 6G, 6I, 6K, or 6M) that was displayed. The detected facialexpression, head position, or head tilt of the user 52 may indicate orat least infer a particular mental state of the user and which may beuseful for, for example, providing subsequent augmented views.

Turning now to FIG. 11E, in the same or alternative implementations, theuser reaction correlating operation 904 may additionally oralternatively include or involve an operation 1151 for correlating thedetected one or more user reactions with one or more forms of the one ormore augmentations included in the augmented view that was presented.For instance, the user reaction associating module 104* of the AR device70* of FIG. 7A or 7B correlating (e.g., linking or associating) thedetected one or more user reactions (e.g., user reactions as expressedthrough the user's eye focus/movement, facial expressions, and so forth)with one or more forms of the one or more augmentations (e.g.,augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h ofFIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) included in the augmented view(e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 mof FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) that was presented.

As further illustrated in FIG. 11E, operation 1151 may further includeone or more additional operations in various alternative implementationsincluding, in some cases, an operation 1152 for correlating the detectedone or more user reactions with one or more shapes and/or dimensions ofthe one or more augmentations. For instance, the user reactionassociating module 104* of the AR device 70* of FIG. 7A or 7Bcorrelating the detected one or more user reactions with one or moreshapes and/or dimensions of the one or more augmentations (e.g.,augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h ofFIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In the same or alternative implementations, operation 1151 mayadditionally or alternatively include an operation 1153 for correlatingthe detected one or more user reactions with one or more locations ofthe one or more augmentations in the augmented view. For instance, theuser reaction associating module 104* of the AR device 70* of FIG. 7A or7B correlating the detected one or more user reactions with one or morelocations of the one or more augmentations (e.g., augmentation 64 a, 64b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F,6G, 6I, 6K, or 6M) in the augmented view (e.g., augmented view 60 b, 60d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I,6K, or 6M).

In some implementations, the user reaction correlating operation 904 mayadditionally or alternatively include or involve an operation 1154 forcorrelating the detected one or more user reactions with the at leastone or more aspects associated with the one or more augmentationsincluding correlating the detected one or more user reactions with oneor more aspects of one or more non-augmentation scene elements includedin the augmented view. For instance, the user reaction associatingmodule 104* of the AR device 70* of FIG. 7A or 7B correlating thedetected one or more user reactions with the at least one or moreaspects associated with the one or more augmentations includingcorrelating (e.g., linking or associating) the detected one or more userreactions (e.g., eye movements) with one or more aspects of one or morenon-augmentation scene elements (e.g., a sign, a building, a street, apicture, and so forth) included in the augmented view (e.g., augmentedview 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG. 6B, 6D,6E, 6F, 6G, 6I, 6K, or 6M).

As further illustrated in FIG. 11E, in some cases, operation 1154 mayfurther include an operation 1155 for correlating the detected one ormore user reactions with the one or more aspects of the one or morenon-augmentation scene elements included in the augmented view bycorrelating the detected one or more user reactions with one or morelocations of the one or more non-augmentation scene elements in theaugmented view. For instance, the user reaction associating module 104*of the AR device 70* of FIG. 7A or 7B correlating the detected one ormore user reactions with the one or more aspects of the one or morenon-augmentation scene elements included in the augmented view bycorrelating the detected one or more user reactions (e.g., eye movementsincluding dwell path of the user 52) with one or more locations of theone or more non-augmentation scene elements in the augmented view (e.g.,augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i, 60 k, or 60 m of FIG.6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

In some cases, operation 1155 may actually include or involve anoperation 1156 for correlating the detected one or more user reactionswith the one or more locations of the one or more non-augmentation sceneelements in the augmented view including correlating the detected one ormore user reactions with the one or more locations of the one or morenon-augmentation scene elements relative to one or more locations of theone or more augmentations in the augmented view. For instance, the userreaction associating module 104* of the AR device 70* of FIG. 7A or 7Bcorrelating the detected one or more user reactions with the one or morelocations of the one or more non-augmentation scene elements in theaugmented view including correlating the detected one or more userreactions with the one or more locations of the one or morenon-augmentation scene elements (e.g., beach, sky, picture, sign, and soforth) relative to one or more locations of the one or moreaugmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e,64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) in theaugmented view (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60 g, 60 i,60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M). Such anoperation may be executed in order to, for example, take into accountsituations where a user 52 behaves or reacts differently depending onwhere non-augmentation scene elements are located in the augmented viewrelative to the location of one or more augmentations in an augmentedview.

In some implementations, the user reaction correlating operation 904 mayadditionally or alternatively include or involve an operation 1157 forcorrelating the detected one or more user reactions with one or morenon-augmentation scene elements that have been obfuscated in theaugmented view by the one or more augmentations. For instance, the userreaction associating module 104* of the AR device 70* of FIG. 7A or 7Bcorrelating (e.g., linking or associating) the detected one or more userreactions (e.g., eye movements, pupil characteristics, skin response,and so forth) with one or more non-augmentation scene elements (e.g., asign, a picture, a store front, and so forth) that have been obfuscatedin the augmented view (e.g., augmented view 60 b, 60 d, 60 e, 60 f, 60g, 60 i, 60 k, or 60 m of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M) by theone or more augmentations (e.g., augmentation 64 a, 64 b, 64 c, 64 cc,64 d, 64 e, 64 f, 64 g, or 64 h of FIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or6M).

Turning now to FIG. 12 illustrating another operational flow 1200.Operational flow 1200 includes certain operations that mirror theoperations included in operational flow 900 of FIG. 9. These operationsinclude a user reaction detecting operation 1202 and a user reactioncorrelating operation 1204 that corresponds to and mirror the userreaction detecting operation 902 and the user reaction correlatingoperation 904, respectively, of FIG. 9.

In addition, operational flow 1200 may include operation 1206 fortransmitting one or more results of the correlating. For instance, thecorrelation result relaying module 107* (e.g., the correlation resultrelaying module 107′ or the correlation result relaying module 107″) ofthe AR device 70* of FIG. 7A or 7B transmitting (e.g., relaying orsending) one or more results of the correlating (e.g., transmitting oneor more results of associating the one or more user reactions with theone or more aspects of one or more augmentations).

As further illustrated in FIG. 12, operation flow 1200 may furtherinclude an operation 1208 for receiving, in response to saidtransmitting, one or more second augmentations for displaying in asecond augmented view of the actual scene or of a second actual scenefrom the real environment. For instance, the augmentation acquiringmodule 108* (e.g., the augmentation acquiring module 108′ or theaugmentation acquiring module 108″) of the AR device 70* of FIG. 7A or7B receiving, in response to said transmitting, one or more secondaugmentations (e.g., augmentation 64 b of FIG. 6D) for displaying in asecond augmented view (e.g., augmented view 60 d of FIG. 6D) of theactual scene or of a second actual scene from the real environment. Forexample, FIG. 6B illustrates an example augmented view 60 b in which asundial augmentation 64 a has been inserted into the augmented view 60b. The eye or eyes of the user 52 viewing the augmented view 60 b maythen be monitored and the dwell path 66 a of the eye or eyes of the user52 may be tracked as illustrated as illustrated in FIG. 6C. Based, onsuch tracking of the dwell path 66 a, another augmentation 64 b may beincluded into an another augmented view 60 d. Note that in someimplementations, the augmented view 64 d may be the augmented view ofessentially the same actual scene at approximately the same point intime. Alternatively, the augmented view 64 d may be the augmented viewof another actual scene (e.g., may be the same scene but at a differentpoint in time) that was generated based on the previously learnedbehavior (e.g., tracked dwell path 66 a) of the user 52 as illustratedin FIG. 6C.

Referring now to FIG. 13, which illustrates how operation 1206 and 1208of FIG. 12 may be implemented in various alternative implementations.For example, in some implementations, operation 1206 may include orinvolve an operation 1359 for transmitting the one or more results ofthe correlating via one or more wireless and/or wired networks. Forinstance, the correlation result relaying module 107* of the AR device70* of FIG. 7A or 7B transmitting the one or more results of thecorrelating via one or more wireless and/or wired networks (e.g., awireless local area network or WLAN, a wide area network or WAN, ametropolitan area network, a cellular network, a public switchedtelephone network or PSTN, and so forth).

In the same or alternative implementations, operation 1206 mayadditionally or alternatively include an operation 1360 for transmittingthe one or more results of the correlating to one or more networkservers associated with one or more independent parties. For instance,the correlation result relaying module 107* of the AR device 70* of FIG.7A or 7B transmitting the one or more results of the correlating to oneor more network servers associated with one or more independent parties(e.g., advertisers, academic institutions, sports arena, retail stores,and so forth).

In the same or alternative implementations, operation 1206 mayadditionally or alternatively include an operation 1361 for transmittingthe one or more results of the correlating by transmitting one or moredefined relationships between the detected one or more user reactionsand the at least one or more aspects associated with the one or moreaugmentations. For instance, the correlation result relaying module 107*of the AR device 70* of FIG. 7A or 7B transmitting the one or moreresults of the correlating by transmitting one or more definedrelationships between the detected one or more user reactions (e.g.,facial expressions or eye movements and focus) and the at least one ormore aspects associated with the one or more augmentations (e.g.,augmentation 64 a, 64 b, 64 c, 64 cc, 64 d, 64 e, 64 f, 64 g, or 64 h ofFIG. 6B, 6D, 6E, 6F, 6G, 6I, 6K, or 6M).

As further illustrated in FIG. 13, operation 1208 for receiving, inresponse to said transmitting, one or more second augmentations fordisplaying in a second augmented view of the actual scene or of a secondactual scene from the real environment may include one or moreadditional operations in various alternative implementations. Forexample, in some implementations, operation 1208 may include anoperation 1362 for receiving the one or more second augmentations forthe displaying including receiving one or more second augmentationshaving one or more formats that were selected based, at least in part,on the one or more results of the correlating that were transmitted. Forinstance, the augmentation acquiring module 108* of the AR device 70* ofFIG. 7A or 7B receiving (e.g., wirelessly acquiring) the one or moresecond augmentations for the displaying including receiving one or moresecond augmentations (e.g., augmentation 64 b of FIG. 6D) having one ormore formats (e.g., an augmentation 64 b in the shape of a soft drinkbottle) that were selected remotely (e.g., selected remotely by a remotenetwork server) based, at least in part, on the one or more results ofthe correlating that were transmitted or relayed.

In some cases, operation 1362 may, in turn, further include an operation1363 for receiving the one or more second augmentations having the oneor more formats including receiving one or more second augmentationsthat are designed to be placed at one or more particular locations inthe second augmented view, the placement or placements of the one ormore second augmentations at the one or more particular locations in thesecond augmented view being based, at least in part, on the one or moreresults of the correlating that were transmitted. For instance, theaugmentation acquiring module 108* of the AR device 70* of FIG. 7A or 7Breceiving the one or more second augmentations having the one or moreformats including receiving one or more second augmentations (e.g.,augmentation 64 b of FIG. 6D) that are designed to be placed at one ormore particular locations (e.g., right bottom corner of the augmentedview 60 d and on top of the beach displayed in the augmented view 60 d)in the second augmented view 60 d (see FIG. 6D), the placement orplacements of the one or more second augmentations 64 b at the one ormore particular locations in the second augmented view 60 d (see FIG.6D) being based, at least in part, on the one or more results of thecorrelating that were transmitted (e.g., the results that have beentransmitted or relayed that indicated the user's dwell path 66 a duringand/or immediately following display of an augmentation 64 a asillustrated in FIG. 6C).

In the same or different implementations, operation 1362 mayadditionally or alternatively include an operation 1364 for receivingthe one or more second augmentations having the one or more formatsincluding receiving one or more second augmentations that are designedto have one or more particular dimensions or shapes, the one or moreparticular dimensions or shapes of the one or more second augmentationsbeing based, at least in part, on the one or more results of thecorrelating that were transmitted. For instance, the augmentationacquiring module 108* of the AR device 70* of FIG. 7A or 7B receivingthe one or more second augmentations having the one or more formatsincluding receiving one or more second augmentations (e.g., augmentation64 b of FIG. 6D) that are designed to have one or more particulardimensions or shapes (e.g., shape of a soft drink bottle), the one ormore particular dimensions or shapes of the one or more secondaugmentations 64 b being based, at least in part, on the one or moreresults of the correlating that were transmitted (e.g., correlation thatlinks a dwell path 66 a of the user 52 with the augmentation 64 a and/orto one or more non-augmentation scene elements such as the beachdisplayed in the actual view 60 b of FIG. 6B and the augmented view 60 cof FIG. 6C).

In the same or different implementations, operation 1208 for receiving,in response to said transmitting, one or more second augmentations fordisplaying in a second augmented view of the actual scene or of a secondactual scene from the real environment may additionally or alternativelyinclude an operation 1365 for receiving the one or more secondaugmentations for the displaying including receiving one or morevisibility rules for displaying the one or more second augmentationsthat are based, at least in part, on the one or more results of thecorrelating that were transmitted. For instance, the augmentationacquiring module 108* of the AR device 70* of FIG. 7A or 7B receivingthe one or more second augmentations for the displaying includingreceiving one or more visibility rules for displaying the one or moresecond augmentations that are based, at least in part, on the one ormore results of the correlating that were transmitted. In variousimplementations, a visibility rule is a directive or command thatdefines when and how an augmentation may be displayed. For example, insome cases, a visibility rule may identify an “anchor” visual pattern oritem (e.g., a non-augmentation scene element) that must be present in anactual scene before the augmentation is to be included in the augmentedview of the actual scene. A visibility rule may additionally oralternatively define the placement of an augmentation in an augmentedview and/or the shape, coloring, and/or dimensions of the augmentation.

Turning now to FIG. 14 illustrating another operational flow 1400.Operational flow 1400 includes certain operations that mirror theoperations included in operational flow 900 of FIG. 9. These operationsinclude a user reaction detecting operation 1402 and a user reactioncorrelating operation 1404 that corresponds to and mirror the userreaction detecting operation 902 and the user reaction correlatingoperation 904, respectively, of FIG. 9.

In addition, operational flow 1400 may further include an operation 1406for displaying a second augmented view of the actual scene or of asecond actual scene from the real environment, the second augmented viewincluding one or more second augmentations that have been included intothe second augmented view based, at least in part, on the correlating.For instance, the augmented view presenting module 106* (e.g., augmentedview presenting module 106′ or the augmented view presenting module106″) of the AR device 70* of FIG. 7A or 7B displaying (e.g., visualpresenting) a second augmented view (e.g., augmented view 60 f of FIG.6F) of the actual scene or of a second actual scene from the realenvironment, the second augmented view 60 f including one or more secondaugmentations (e.g., augmentation 64 d) that have been included into thesecond augmented view 60 f (see FIG. 6F) based, at least in part, on thecorrelating (e.g., the correlation or association of the tracked dwellpath 66 b of the user 52, as illustrated in FIG. 6E, with anaugmentation 64 c of FIGS. 6E and 6F).

In some cases, operation 1406 may further include or involve anoperation 1467 for displaying the second augmented view of the actualscene or of the second actual scene from the real environment thatincludes the one or more second augmentations including displaying asecond augmented view that includes one or more second augmentationshaving one or more formats that were selected based, at least in part,on the one or more results of the correlating as further illustrated inFIG. 14. For instance, the augmented view presenting module 106* of theAR device 70* of FIG. 7A or 7B displaying the second augmented view ofthe actual scene or of the second actual scene from the real environmentthat includes the one or more second augmentations including displaying(e.g. visually presenting) a second augmented view (e.g., augmented view60 f of FIG. 6F) that includes one or more second augmentations (e.g.,augmentation 64 d of FIG. 6F) having one or more formats (e.g., a visualimage of a plane pulling a banner as illustrated in FIG. 6F) that wereselected based, at least in part, on the one or more results of thecorrelating (e.g., associating of the one or more user reactions withthe at least one or more aspects of the one or more augmentations).

In various implementations, operation 1467 may further include one ormore additional operations including, in some cases, an operation 1468for displaying the second augmented view that includes the one or moresecond augmentations having the one or more formats by displaying asecond augmented view that includes one or more second augmentationsthat are selectively placed at one or more particular locations in thesecond augmented view, the one or more placements of the one or moresecond augmentations at the one or more particular locations in thesecond augmented view being based, at least in part, on the correlating.For instance, the augmented view presenting module 106* of the AR device70* of FIG. 7A or 7B displaying (e.g., visually presenting) the secondaugmented view that includes the one or more second augmentations havingthe one or more formats by displaying a second augmented view (e.g.,augmented view 60 f of FIG. 6F) that includes one or more secondaugmentations 64 d (see FIG. 6F) that are selectively placed at one ormore particular locations in the second augmented view 60 f (e.g., topright corner and in the sky depicted in the augmented view 60 f of FIG.6F), the one or more placements of the one or more second augmentationsat the one or more particular locations in the second augmented view 60f being based, at least in part, on the correlating.

In the same or alternative implementations, operation 1467 mayadditionally or alternatively include an operation 1469 for displayingthe second augmented view that includes the one or more secondaugmentations having the one or more formats by displaying a secondaugmented view that includes one or more second augmentations that haveone or more particular dimensions or shapes, the one or more particulardimensions or shapes of the one or more second augmentations beingbased, at least in part, on the correlating. For instance, the augmentedview presenting module 106* of the AR device 70* of FIG. 7A or 7Bdisplaying the second augmented view that includes the one or moresecond augmentations having the one or more formats by displaying asecond augmented view (e.g. augmented view 60 f of FIG. 6F) thatincludes one or more second augmentations (e.g., augmentation 64 d) thathave one or more particular dimensions or shapes (e.g., in the shape ofa plane pulling an advertising banner), the one or more particulardimensions or shapes of the one or more second augmentations 64 d beingbased, at least in part, on the correlating.

In various implementations, operation 1406 for displaying a secondaugmented view of the actual scene or of a second actual scene from thereal environment, the second augmented view including one or more secondaugmentations that have been included into the second augmented viewbased, at least in part, on the correlating may include an operation1470 for displaying the second augmented view of the actual scene or ofthe second actual scene from the real environment that includes the oneor more second augmentations including displaying a second augmentedview of the actual scene that includes one or more second augmentationsthat replaces the one or more augmentations in the second augmentedview. For instance, the augmented view presenting module 106* of the ARdevice 70* of FIG. 7A or 7B displaying the second augmented view of theactual scene or of the second actual scene from the real environmentthat includes the one or more second augmentations including displayinga second augmented view (e.g., augmented view 60 m of FIG. 6M) of theactual scene that includes one or more second augmentations (e.g.,augmentation 64 h of FIG. 6H) that replaces the one or moreaugmentations (e.g., augmentation 64 f of FIG. 6I) in the secondaugmented view 60 m (see FIG. 6M). For example, FIG. 6I illustrated anaugmented view 60 i of an actual scene (e.g., actual view 60 h of FIG.6H) from a shopping mall. The augmented view 60 i includes anaugmentation 64 f that provides advertisement information related to aparticular restaurant in the shopping mall. Based, for example, on therelatively small dwell time of the user 52 (e.g., which indicates thatthe user has little interest) as it relates to the displayedaugmentation 64 f, the next time the user 52 views the same actual scene(e.g., actual view 60 h of FIG. 6H), the AR device 70* may generate asecond augmented view (e.g., augmented view 60 m of FIG. 6M) thatincludes an augmentation 64 h that replaces the original augmentation 64f, the replacement augmentation 64 h having a different border and colorin the hopes of increasing the dwell time of the user 52 (e.g., aboarder and color that are intended to entice the interest of the user52).

In a general sense, those skilled in the art will recognize that thevarious aspects described herein which can be implemented, individuallyand/or collectively, by a wide range of hardware, software, firmware,and/or any combination thereof can be viewed as being composed ofvarious types of “electrical circuitry.” Consequently, as used herein“electrical circuitry” includes, but is not limited to, electricalcircuitry having at least one discrete electrical circuit, electricalcircuitry having at least one integrated circuit, electrical circuitryhaving at least one application specific integrated circuit, electricalcircuitry forming a general purpose computing device configured by acomputer program (e.g., a general purpose computer configured by acomputer program which at least partially carries out processes and/ordevices described herein, or a microprocessor configured by a computerprogram which at least partially carries out processes and/or devicesdescribed herein), electrical circuitry forming a memory device (e.g.,forms of memory (e.g., random access, flash, read only, etc.)), and/orelectrical circuitry forming a communications device (e.g., a modem,communications switch, optical-electrical equipment, etc.). Those havingskill in the art will recognize that the subject matter described hereinmay be implemented in an analog or digital fashion or some combinationthereof.

It has been argued that because high-level programming languages usestrong abstraction (e.g., that they may resemble or share symbols withnatural languages), they are therefore a “purely mental construct.”(e.g., that “software”—a computer program or computer programming—issomehow an ineffable mental construct, because at a high level ofabstraction, it can be conceived and understood in the human mind). Thisargument has been used to characterize technical description in the formof functions/operations as somehow “abstract ideas.” In fact, intechnological arts (e.g., the information and communicationtechnologies) this is not true.

The fact that high-level programming languages use strong abstraction tofacilitate human understanding should not be taken as an indication thatwhat is expressed is an abstract idea. In fact, those skilled in the artunderstand that just the opposite is true. If a high-level programminglanguage is the tool used to implement a technical disclosure in theform of functions/operations, those skilled in the art will recognizethat, far from being abstract, imprecise, “fuzzy,” or “mental” in anysignificant semantic sense, such a tool is instead a nearincomprehensibly precise sequential specification of specificcomputational machines—the parts of which are built up byactivating/selecting such parts from typically more generalcomputational machines over time (e.g., clocked time). This fact issometimes obscured by the superficial similarities between high-levelprogramming languages and natural languages. These superficialsimilarities also may cause a glossing over of the fact that high-levelprogramming language implementations ultimately perform valuable work bycreating/controlling many different computational machines.

The many different computational machines that a high-level programminglanguage specifies are almost unimaginably complex. At base, thehardware used in the computational machines typically consists of sometype of ordered matter (e.g., traditional electronic devices (e.g.,transistors), deoxyribonucleic acid (DNA), quantum devices, mechanicalswitches, optics, fluidics, pneumatics, optical devices (e.g., opticalinterference devices), molecules, etc.) that are arranged to form logicgates. Logic gates are typically physical devices that may beelectrically, mechanically, chemically, or otherwise driven to changephysical state in order to create a physical reality of Boolean logic.

Logic gates may be arranged to form logic circuits, which are typicallyphysical devices that may be electrically, mechanically, chemically, orotherwise driven to create a physical reality of certain logicalfunctions. Types of logic circuits include such devices as multiplexers,registers, arithmetic logic units (ALUs), computer memory, etc., eachtype of which may be combined to form yet other types of physicaldevices, such as a central processing unit (CPU)—the best known of whichis the microprocessor. A modern microprocessor will often contain morethan one hundred million logic gates in its many logic circuits (andoften more than a billion transistors). See, e.g., Wikipedia, Logicgates, http://en.wikipedia.org/wiki/Logic_gates (as of Jun. 5, 2012,21:03 GMT).

The logic circuits forming the microprocessor are arranged to provide amicroarchitecture that will carry out the instructions defined by thatmicroprocessor's defined Instruction Set Architecture. The InstructionSet Architecture is the part of the microprocessor architecture relatedto programming, including the native data types, instructions,registers, addressing modes, memory architecture, interrupt andexception handling, and external Input/Output. See, e.g., Wikipedia,Computer architecture,http://en.wikipedia.org/wiki/Computer_architecture (as of Jun. 5, 2012,21:03 GMT).

The Instruction Set Architecture includes a specification of the machinelanguage that can be used by programmers to use/control themicroprocessor. Since the machine language instructions are such thatthey may be executed directly by the microprocessor, typically theyconsist of strings of binary digits, or bits. For example, a typicalmachine language instruction might be many bits long (e.g., 32, 64, or128 bit strings are currently common). A typical machine languageinstruction might take the form “11110000101011110000111100111111” (a 32bit instruction).

It is significant here that, although the machine language instructionsare written as sequences of binary digits, in actuality those binarydigits specify physical reality. For example, if certain semiconductorsare used to make the operations of Boolean logic a physical reality, theapparently mathematical bits “1” and “0” in a machine languageinstruction actually constitute a shorthand that specifies theapplication of specific voltages to specific wires. For example, in somesemiconductor technologies, the binary number “1” (e.g., logical “1”) ina machine language instruction specifies around +5 volts applied to aspecific “wire” (e.g., metallic traces on a printed circuit board) andthe binary number “0” (e.g., logical “0”) in a machine languageinstruction specifies around −5 volts applied to a specific “wire.” Inaddition to specifying voltages of the machines' configuration, suchmachine language instructions also select out and activate specificgroupings of logic gates from the millions of logic gates of the moregeneral machine. Thus, far from abstract mathematical expressions,machine language instruction programs, even though written as a stringof zeros and ones, specify many, many constructed physical machines orphysical machine states.

Machine language is typically incomprehensible by most humans (e.g., theabove example was just ONE instruction, and some personal computersexecute more than two billion instructions every second). See, e.g.,Wikipedia, Instructions per second,http://en.wikipedia.org/wiki/Instructions_per_second (as of Jun. 5,2012, 21:04 GMT). Thus, programs written in machine language—which maybe tens of millions of machine language instructions long—areincomprehensible. In view of this, early assembly languages weredeveloped that used mnemonic codes to refer to machine languageinstructions, rather than using the machine language instructions'numeric values directly (e.g., for performing a multiplicationoperation, programmers coded the abbreviation “mult,” which representsthe binary number “011000” in MIPS machine code). While assemblylanguages were initially a great aid to humans controlling themicroprocessors to perform work, in time the complexity of the work thatneeded to be done by the humans outstripped the ability of humans tocontrol the microprocessors using merely assembly languages.

At this point, it was noted that the same tasks needed to be done overand over, and the machine language necessary to do those repetitivetasks was the same. In view of this, compilers were created. A compileris a device that takes a statement that is more comprehensible to ahuman than either machine or assembly language, such as “add 2+2 andoutput the result,” and translates that human understandable statementinto a complicated, tedious, and immense machine language code (e.g.,millions of 32, 64, or 128 bit length strings). Compilers thus translatehigh-level programming language into machine language.

This compiled machine language, as described above, is then used as thetechnical specification which sequentially constructs and causes theinteroperation of many different computational machines such thathumanly useful, tangible, and concrete work is done. For example, asindicated above, such machine language—the compiled version of thehigher-level language—functions as a technical specification whichselects out hardware logic gates, specifies voltage levels, voltagetransition timings, etc., such that the humanly useful work isaccomplished by the hardware.

Thus, a functional/operational technical description, when viewed by oneof skill in the art, is far from an abstract idea. Rather, such afunctional/operational technical description, when understood throughthe tools available in the art such as those just described, is insteadunderstood to be a humanly understandable representation of a hardwarespecification, the complexity and specificity of which far exceeds thecomprehension of most any one human. With this in mind, those skilled inthe art will understand that any such operational/functional technicaldescriptions—in view of the disclosures herein and the knowledge ofthose skilled in the art—may be understood as operations made intophysical reality by (a) one or more interchained physical machines, (b)interchained logic gates configured to create one or more physicalmachine(s) representative of sequential/combinatorial logic(s), (c)interchained ordered matter making up logic gates (e.g., interchainedelectronic devices (e.g., transistors), DNA, quantum devices, mechanicalswitches, optics, fluidics, pneumatics, molecules, etc.) that createphysical reality representative of logic(s), or (d) virtually anycombination of the foregoing. Indeed, any physical object which has astable, measurable, and changeable state may be used to construct amachine based on the above technical description. Charles Babbage, forexample, constructed the first computer out of wood and powered bycranking a handle.

Thus, far from being understood as an abstract idea, those skilled inthe art will recognize a functional/operational technical description asa humanly-understandable representation of one or more almostunimaginably complex and time sequenced hardware instantiations. Thefact that functional/operational technical descriptions might lendthemselves readily to high-level computing languages (or high-levelblock diagrams for that matter) that share some words, structures,phrases, etc. with natural language simply cannot be taken as anindication that such functional/operational technical descriptions areabstract ideas, or mere expressions of abstract ideas. In fact, asoutlined herein, in the technological arts this is simply not true. Whenviewed through the tools available to those of skill in the art, suchfunctional/operational technical descriptions are seen as specifyinghardware configurations of almost unimaginable complexity.

As outlined above, the reason for the use of functional/operationaltechnical descriptions is at least twofold. First, the use offunctional/operational technical descriptions allows near-infinitelycomplex machines and machine operations arising from interchainedhardware elements to be described in a manner that the human mind canprocess (e.g., by mimicking natural language and logical narrativeflow). Second, the use of functional/operational technical descriptionsassists the person of skill in the art in understanding the describedsubject matter by providing a description that is more or lessindependent of any specific vendor's piece(s) of hardware.

The use of functional/operational technical descriptions assists theperson of skill in the art in understanding the described subject mattersince, as is evident from the above discussion, one could easily,although not quickly, transcribe the technical descriptions set forth inthis document as trillions of ones and zeroes, billions of single linesof assembly-level machine code, millions of logic gates, thousands ofgate arrays, or any number of intermediate levels of abstractions.However, if any such low-level technical descriptions were to replacethe present technical description, a person of skill in the art couldencounter undue difficulty in implementing the disclosure, because sucha low-level technical description would likely add complexity without acorresponding benefit (e.g., by describing the subject matter utilizingthe conventions of one or more vendor-specific pieces of hardware).Thus, the use of functional/operational technical descriptions assiststhose of skill in the art by separating the technical descriptions fromthe conventions of any vendor-specific piece of hardware.

In view of the foregoing, the logical operations/functions set forth inthe present technical description are representative of static orsequenced specifications of various ordered-matter elements, in orderthat such specifications may be comprehensible to the human mind andadaptable to create many various hardware configurations. The logicaloperations/functions disclosed herein should be treated as such, andshould not be disparagingly characterized as abstract ideas merelybecause the specifications they represent are presented in a manner thatone of skill in the art can readily understand and apply in a mannerindependent of a specific vendor's hardware implementation.

Those having skill in the art will recognize that the state of the arthas progressed to the point where there is little distinct ion leftbetween hardware, software, and/or firmware implementations of aspectsof systems; the use of hardware, software, and/or firmware is generally(but not always, in that in certain contexts the choice between hardwareand software can become significant) a design choice representing costvs. efficiency tradeoffs. Those having skill in the art will appreciatethat there are various vehicles by which processes and/or systems and/orother technologies described herein can be effected (e.g., hardware,software, and/or firmware), and that the preferred vehicle will varywith the context in which the processes and/or systems and/or othertechnologies are deployed. For example, if an implementer determinesthat speed and accuracy are paramount, the implementer may opt for amainly hardware and/or firmware vehicle; alternatively, if flexibilityis paramount, the implementer may opt for a mainly softwareimplementation; or, yet again alternatively, the implementer may opt forsome combination of hardware, software, and/or firmware in one or moremachines, compositions of matter, and articles of manufacture, limitedto patentable subject matter under 35 USC 101. Hence, there are severalpossible vehicles by which the processes and/or devices and/or othertechnologies described herein may be effected, none of which isinherently superior to the other in that any vehicle to be utilized is achoice dependent upon the context in which the vehicle will be deployedand the specific concerns (e.g., speed, flexibility, or predictability)of the implementer, any of which may vary. Those skilled in the art willrecognize that optical aspects of implementations will typically employoptically-oriented hardware, software, and or firmware.

In some implementations described herein, logic and similarimplementations may include software or other control structures.Electronic circuitry, for example, may have one or more paths ofelectrical current constructed and arranged to implement variousfunctions as described herein. In some implementations, one or moremedia may be configured to bear a device-detectable implementation whensuch media hold or transmit device detectable instructions operable toperform as described herein. In some variants, for example,implementations may include an update or modification of existingsoftware or firmware, or of gate arrays or programmable hardware, suchas by performing a reception of or a transmission of one or moreinstructions in relation to one or more operations described herein.Alternatively or additionally, in some variants, an implementation mayinclude special-purpose hardware, software, firmware components, and/orgeneral-purpose components executing or otherwise invokingspecial-purpose components. Specifications or other implementations maybe transmitted by one or more instances of tangible transmission mediaas described herein, optionally by packet transmission or otherwise bypassing through distributed media at various times.

Alternatively or additionally, implementations may include executing aspecial-purpose instruction sequence or invoking circuitry for enabling,triggering, coordinating, requesting, or otherwise causing one or moreoccurrences of virtually any functional operations described herein. Insome variants, operational or other logical descriptions herein may beexpressed as source code and compiled or otherwise invoked as anexecutable instruction sequence. In some contexts, for example,implementations may be provided, in whole or in part, by source code,such as C++, or other code sequences. In other implementations, sourceor other code implementation, using commercially available and/ortechniques in the art, may be compiled/implemented/translated/convertedinto a high-level descriptor language (e.g., initially implementingdescribed technologies in C or C++ programming language and thereafterconverting the programming language implementation into alogic-synthesizable language implementation, a hardware descriptionlanguage implementation, a hardware design simulation implementation,and/or other such similar mode(s) of expression). For example, some orall of a logical expression (e.g., computer programming languageimplementation) may be manifested as a Verilog-type hardware description(e.g., via Hardware Description Language (HDL) and/or Very High SpeedIntegrated Circuit Hardware Descriptor Language (VHDL)) or othercircuitry model which may then be used to create a physicalimplementation having hardware (e.g., an Application Specific IntegratedCircuit). Those skilled in the art will recognize how to obtain,configure, and optimize suitable transmission or computational elements,material supplies, actuators, or other structures in light of theseteachings.

While particular aspects of the present subject matter described hereinhave been shown and described, it will be apparent to those skilled inthe art that, based upon the teachings herein, changes and modificationsmay be made without departing from the subject matter described hereinand its broader aspects and, therefore, the appended claims are toencompass within their scope all such changes and modifications as arewithin the true spirit and scope of the subject matter described herein.It will be understood by those within the art that, in general, termsused herein, and especially in the appended claims (e.g., bodies of theappended claims) are generally intended as “open” terms (e.g., the term“including” should be interpreted as “including but not limited to,” theterm “having” should be interpreted as “having at least,” the term“includes” should be interpreted as “includes but is not limited to,”etc.).

It will be further understood by those within the art that if a specificnumber of an introduced claim recitation is intended, such an intentwill be explicitly recited in the claim, and in the absence of suchrecitation no such intent is present. For example, as an aid tounderstanding, the following appended claims may contain usage of theintroductory phrases “at least one” and “one or more” to introduce claimrecitations. However, the use of such phrases should not be construed toimply that the introduction of a claim recitation by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim recitation to claims containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should typically be interpreted to mean “atleast one” or “one or more”); the same holds true for the use ofdefinite articles used to introduce claim recitations. In addition, evenif a specific number of an introduced claim recitation is explicitlyrecited, those skilled in the art will recognize that such recitationshould typically be interpreted to mean at least the recited number(e.g., the bare recitation of “two recitations,” without othermodifiers, typically means at least two recitations, or two or morerecitations).

Furthermore, in those instances where a convention analogous to “atleast one of A, B, and C, etc.” is used, in general such a constructionis intended in the sense one having skill in the art would understandthe convention (e.g., “a system having at least one of A, B, and C”would include but not be limited to systems that have A alone, B alone,C alone, A and B together, A and C together, B and C together, and/or A,B, and C together, etc.). In those instances where a conventionanalogous to “at least one of A, B, or C, etc.” is used, in general sucha construction is intended in the sense one having skill in the artwould understand the convention (e.g., “a system having at least one ofA, B, or C” would include but not be limited to systems that have Aalone, B alone, C alone, A and B together, A and C together, B and Ctogether, and/or A, B, and C together, etc.). It will be furtherunderstood by those within the art that typically a disjunctive wordand/or phrase presenting two or more alternative terms, whether in thedescription, claims, or drawings, should be understood to contemplatethe possibilities of including one of the terms, either of the terms, orboth terms unless context dictates otherwise. For example, the phrase “Aor B” will be typically understood to include the possibilities of “A”or “B” or “A and B.”

With respect to the appended claims, those skilled in the art willappreciate that recited operations therein may generally be performed inany order. Also, although various operational flows are presented in asequence(s), it should be understood that the various operations may beperformed in other orders than those which are illustrated, or may beperformed concurrently. Examples of such alternate orderings may includeoverlapping, interleaved, interrupted, reordered, incremental,preparatory, supplemental, simultaneous, reverse, or other variantorderings, unless context dictates otherwise. Furthermore, terms like“responsive to,” “related to,” or other past-tense adjectives aregenerally not intended to exclude such variants, unless context dictatesotherwise.

This application may make reference to one or more trademarks, e.g., aword, letter, symbol, or device adopted by one manufacturer or merchantand used to identify and/or distinguish his or her product from those ofothers. Trademark names used herein are set forth in such language thatmakes clear their identity, that distinguishes them from commondescriptive nouns, that have fixed and definite meanings, or, in many ifnot all cases, are accompanied by other specific identification usingterms not covered by trademark. In addition, trademark names used hereinhave meanings that are well-known and defined in the literature, or donot refer to products or compounds for which knowledge of one or moretrade secrets is required in order to divine their meaning. Alltrademarks referenced in this application are the property of theirrespective owners, and the appearance of one or more trademarks in thisapplication does not diminish or otherwise adversely affect the validityof the one or more trademarks. All trademarks, registered orunregistered, that appear in this application are assumed to include aproper trademark symbol, e.g., the circle R or bracketed capitalization(e.g., [trademark name]), even when such trademark symbol does notexplicitly appear next to the trademark. To the extent a trademark isused in a descriptive manner to refer to a product or process, thattrademark should be interpreted to represent the corresponding productor process as of the date of the filing of this patent application.

Those skilled in the art will appreciate that the foregoing specificexemplary processes and/or devices and/or technologies arerepresentative of more general processes and/or devices and/ortechnologies taught elsewhere herein, such as in the claims filedherewith and/or elsewhere in the present application.

1. A computationally-implemented method, comprising: detecting one ormore user reactions of a user in response to a display to the user of anaugmented view of an actual scene from a real environment, the augmentedview that was displayed including one or more augmentations; andcorrelating the detected one or more user reactions with at least one ormore aspects associated with the one or more augmentations that wereincluded in the augmented view that was presented.
 2. Thecomputationally-implemented method of claim 1, wherein said detectingone or more user reactions of a user in response to a display to theuser of an augmented view of an actual scene from a real environment,the augmented view that was displayed including one or moreaugmentations comprises: detecting the one or more user reactions of theuser including sensing one or more physiological characteristics of theuser proximate to the display of the augmented view. 3-20. (canceled)21. The computationally-implemented method of claim 1, wherein saiddetecting one or more user reactions of a user in response to a displayto the user of an augmented view of an actual scene from a realenvironment, the augmented view that was displayed including one or moreaugmentations comprises: detecting the one or more user reactions of theuser including detecting at least one of facial expression, headposition, or head tilt of the user proximate to the display of theaugmented view.
 22. (canceled)
 23. The computationally-implementedmethod of claim 1 wherein said detecting one or more user reactions of auser in response to a display to the user of an augmented view of anactual scene from a real environment, the augmented view that wasdisplayed including one or more augmentations comprises: detecting theone or more user reactions of the user in response to the display of theaugmented view including registering one or more aspects of theaugmented view. 24-30. (canceled)
 31. The computationally-implementedmethod of claim 1, wherein said correlating the detected one or moreuser reactions with at least one or more aspects associated with the oneor more augmentations that were included in the augmented view that waspresented comprises: correlating the detected one or more usercharacteristics with the at least one or more aspects associated withthe one or more augmentations by defining one or more relationshipsbetween the detected one or more user reactions and the at least one ormore aspects associated with the one or more augmentations. 32.(canceled)
 33. The computationally-implemented method of claim 1,wherein said correlating the detected one or more user reactions with atleast one or more aspects associated with the one or more augmentationsthat were included in the augmented view that was presented comprises:correlating the detected one or more user reactions with the at leastone or more aspects associated with the one or more augmentations bycorrelating one or more physiological characteristics of the user thatwere sensed during and/or following the display of the augmented viewwith the one or more aspects of the one or more augmentations. 34-49.(canceled)
 50. The computationally-implemented method of claim 1,wherein said correlating the detected one or more user reactions with atleast one or more aspects associated with the one or more augmentationsthat were included in the augmented view that was presented comprises:correlating the detected one or more user reactions with at least theone or more aspects associated with the one or more augmentations bycorrelating at least one of a facial expression, a head position, or ahead tilt of the user that was detected during and/or following thedisplay of the augmented view with the one or more augmentations thatwere included in the augmented view that was displayed.
 51. Thecomputationally-implemented method of claim 1, wherein said correlatingthe detected one or more user reactions with at least one or moreaspects associated with the one or more augmentations that were includedin the augmented view that was presented comprises: correlating thedetected one or more user reactions with one or more forms of the one ormore augmentations included in the augmented view that was presented.52-53. (canceled)
 54. The computationally-implemented method of claim 1,wherein said correlating the detected one or more user reactions with atleast one or more aspects associated with the one or more augmentationsthat were included in the augmented view that was presented comprises:correlating the detected one or more user reactions with the at leastone or more aspects associated with the one or more augmentationsincluding correlating the detected one or more user reactions with oneor more aspects of one or more non-augmentation scene elements includedin the augmented view. 55-56. (canceled)
 57. Thecomputationally-implemented method of claim 1, wherein said correlatingthe detected one or more user reactions with at least one or moreaspects associated with the one or more augmentations that were includedin the augmented view that was presented comprises: correlating thedetected one or more user reactions with one or more non-augmentationscene elements that have been obfuscated in the augmented view by theone or more augmentations.
 58. The computationally-implemented method ofclaim 1, further comprising: transmitting one or more results of thecorrelating; and receiving, in response to said transmitting, one ormore second augmentations for displaying in a second augmented view ofthe actual scene or of a second actual scene from the real environment.59-65. (canceled)
 66. The computationally-implemented method of claim 1,further comprising: displaying a second augmented view of the actualscene or of a second actual scene from the real environment, the secondaugmented view including one or more second augmentations that have beenincluded into the second augmented view based, at least in part, on thecorrelating. 67-70. (canceled)
 71. A computationally-implemented system,comprising: means for detecting one or more user reactions of a user inresponse to a display to the user of an augmented view of an actualscene from a real environment, the augmented view that was displayedincluding one or more augmentations; and means for correlating thedetected one or more user reactions with at least one or more aspectsassociated with the one or more augmentations that were included in theaugmented view that was presented. 72-140. (canceled)
 141. A system,comprising: circuitry for detecting one or more user reactions of a userin response to a display to the user of an augmented view of an actualscene from a real environment, the augmented view that was displayedincluding one or more augmentations; and circuitry for correlating thedetected one or more user reactions with at least one or more aspectsassociated with the one or more augmentations that were included in theaugmented view that was presented.